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CHAPTER I 


INTRODUCTION 


The Problem 
"It is becoming increasingly obvious that 

the full potential of information processing 

will not and cannot be realized until computers 

and their applications can be managed reliably 

in an economic sense (Weinwurm, 1968, p. 329)." 

Weinwurm's above comment at the 1968 National Conference 
of the Association for Computing Machinery could well be taken 
as the underlying imperative for this research. Computers have 
had great impact on our economy, on organizations, and on 
individuals in the past fifteen years. They have been cursed 
and praised. The literature of virtually every discipline 
contains increasing references to computers. 

However, one recurrent theme that keeps appearing over 
and over again when computerized information systems are under 
discussion is the failure of organizational managements to 
capitalize on the potential offered by computer technology. 
Many accounts of such management shortcomings could be cited, 
and some will be cited further on, but Reynolds (1968) summed 
up the essence of all of them quite succinctly when he said: 

"The common complaint among people who must 
plan for and manage the development of computer 
program systems is that the products are almost 


always finished over budget and late, and they 
hardly ever do what they were intended to do (p. 334)." 
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John Diebold (1969), a pioneer in the information 
systems field, has predicted that "by 1985, the computer will 
have become central to the nervous system of the corporation," 
Diebold (1969) has also pointed out that the "share of personnel 
and software cost in the total ADP mix'' has substantially 
increased in the past five years to the point where these costs 
often amount to twice the annual hardware cost. Both of these 
factors lead one to the conclusion that the management of 
management information systems (MIS) should be the focus of 


much attention and research. 


The Need 
Before proceeding to a statement of the purpose for 
the present research, the following rather extensive quotation 
from Weinwurm (1968) emphasizes the need for the type of research 
conducted by the author and reported in this thesis: 


"Of the explosive and unremitting change that is 
characteristic of information processing there 
can be no doubt. Nonetheless, there is a great 
difference between noting the pace and dynanism 
of the field and asserting that, as a result, 
management research is bound to be wasteful-- 

no matter what the investment in it, regardless 
of the probability of success, and whatever the 
savings that may result. The empirical evidence 
that can be marshalled in support of such a 
compound assertion is rather meagre. Indeed, it 
is doubtful whether a deliberate policy of 
investing in management research on a scale that 
is in some substantive sense proportional to the 
magnitude of technical development has ever been 
seriously attempted. 
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"Similarly since all possible innovations and 
contingencies obviously cannot be anticipated, 
learning by experience is a necessary concomi- 
tant to any technological advance. Adopting 
‘trial and error’ as a governing policy is 
another matter altogether. There is consider- 
able reason to believe that the real costs of 
information processing ‘'trials' are far, far 
higher than is generally recognized. In fact, 
due to the ingenious ways by means of which we 
have learned to obfuscate learning inefficiencies, 
the real costs are very likely indeterminate. 
In any case, no serious attempt has ever been 
made to demonstrate that ‘trial and error' is 
the most economically effective--or the most 
expeditious--way of advancing the state of the 
management art; perhaps because such a demon- 
stration may well involve contradictions of a 
rather fundamental nature. 


"Finally, there are undoubtedly rather basic 
similarities between information processing and 
any other process that involves men and machines. 
At the same time, these resemblances can be 
carried beyond the point of supportability. 

While the general validity of certain principles 
of good and effective management can be assumed, 
information processing in many respects differs 
from the manufacturing processes upon which much 
of our management knowledge is based. To the 
extent of these differences--which are expecially 
evident, for instance, in computer programming-<- 
literal and indiscriminate application of tradi- 
tional management principles can on occasion evoke 
rather preposterous management policies. Hence, 
the critical issue is not whether well-established 
management methods are entirely relevant or 
whether completely new variations must be developed, 
but which combination of the old and the new is 
most appropriate to and effective under each 
circumstance. 


"In any case, whever the ‘truth', collectively, 
lies, one would expect these differences of 
opinion to be discussed by the leading experts, 
with the support of a reasonable assortment of 
facts instead of assertions, in and through the 
formal meetings and journals that the relevant 
societies provide for such purposes. That, 





after all, is the professional and scientific 
way such issues are resolved. Regrettably, 
very little if anything of this kind has 
happened. It is true that there have been 
hundreds, and probably thousands, of discussions, 
symposia, articles, tutorials and the like, 
during the past 10 years, but nearly all have 
been on a transparently superficial, often 
amateurish and promotional level. And one 
finds it difficult to hear of comprehensive, 
comparable, or reliably-analyzed experience- 
data--let alone the management measures and 
standards that derive from such data--anywhere. 


"I believe that we cannot abide this situation 
any longer. It is high time that the management 
of information processing was brought into the 
mainstream of professional discussion; i.e., 
that theses be formally advanced and supported 
by facts in a respectable manner; and that 

this be done ‘on the record’ so that others 

will be stimulated to counter in the same 
fashion," 
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"While the lack of a generally available, 
accepted, or applicable management methodology 
is a serious problem in nearly every area of 
information processing, I believe most 

experts would agree that the greatest con- 
fusion and the most critical need is in the 
management of computer programming--in the 
broad sense of the term. Moreover, software- 
related problems--by every present indication-<- 
will increase far more rapidly in their scope, 
complexity and economic impact than will their 
hardware counterparts during the forseeable 
future (pp. 330-331)." 


Purpose 


The purpose of the research detailed in this thesis 
can perhaps best be summarized by the question: 


What organizational and procedural factors are 
correlates of success with MIS projects? 
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Raising such a question and pursuing its answers are predicated 
on the hypothesis that there are certain organizational and 
procedural variables which have a meaningful influence on the 
success or failure of a given MIS project, and that these 
variables can be identified. Further, upon identification, 
such variables can be manipulated in certain ways under certain 
conditions to enhance the probability of success with a given 


MIS project. 


Definition of MIS 

Before going further, it is necessary to define just 
what is meant by_a "management information system'’ since there 
does not appear to be much consensus on a definition among 
either authors or practitioners in the field (Dickson, 1970). 
One can arbitrarily designate any system or method that provides 
information for the decision-making functions of management as 
a MIS. A great many sources of management information beyond 
the scope of the usage in this thesis would thus qualify as a 
MIS. (For example, the Wall Street Journal.) However, the 
term MIS has only come to be used extensively since the advent 
of electronic computers for processing information to aid 
managerial decision-making. In fact, it would appear that a 
great many people equate MIS and the computer. 

It is this latter equation that the author considers 
to be incorrect. There have been, and are today, many kinds of 


computer applications which should be classified as data processing 
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applications as opposed to MIS applications. The perennial 
favorite, when such a distinction is raised, is a payroll 
application which is merely converted from one form of process- 
ing (manual or punched card) to some other form (computer). 
Another example of a non-MIS computer application would be an 
inventory-control system which merely does on the computer what 
stock clerks were doing with card files, and nothing more. 
However, the inventory-control application could become a MIS 
if upgraded to the point that summarized or analyzed data were 
made available to management for the express purpose of making 
decisions on such questions as balancing workloads, make or 
buy, and optimal forms of distribution to minimize cost and 
maximize service. The last example might be viewed as the 
threshold for qualifying as a MIS application, since some of 
the kinds of decisions cited could be programmed and handled 

by the computer under certain circumstances. The important 
distinction, then, is that for a computer application to be 
considered a MIS application, a manager at some level (not just 
the corporate executives) must be receiving information from 
the computer which aids him in making decisions within his 
domain of responsibility. Such decisions could be of either 


a planning nature or a control nature, or both. 


The Focus 
Three additional points require elaboration with 


respect to the stated purpose of this research. First, the 
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focus of the research reported here, or "element", using the 
terminology of Lazarsfeld and Menzel (1961), is the MIS project. 
This is a distinct departure from previous empirical studies 
of various aspects of computer usage in organizations. Generally, 
the element studied has been the overall organization or some 
segment of the organization. For example, there has been 
considerable attention devoted to the impact of computers on 
the organization (Shultz and Whisler, 1960; Myers, 1967; Reif, 
1968). Another approach used in studying the organizational 
implications of computers has been to use the information systems 
function, or “computer complex", as the element (Reichenbach & 
Tasso, 1968). This latter method involves trying to evaluate 
the impact of various organizational variables on the effective- 
ness of the overall computer effort. 

The assumption here is that if methods can be found 
to increase the probability of success with specific MIS projects, 
the overall success of the total MIS effort will logically follow. 
While this is a strong assumption, it is further assumed that 
an organization doing a good job on a project by project basis 
will not completely neglect longerange MIS planning and integra- 
tion. At any rate, it is believed that more meaningful hypotheses 
can be posed and tested for the individual MIS project than for 
the entire MIS effort of an organization. Further, there is 
need to evaluate the effectiveness of given practices in more 


discrete situational contexts than the total organization, 
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since project characteristics may possibly vary in such a way 
as to call for differential treatments. 

The second point requiring elaboration follows 
directly from the definition given of the MIS project, and is 
related to the point about project orientation. As noted, most 
previous studies have dealt with the total organization in 
looking at MIS success, and in doing so have encompassed an 
indeterminable amount of information system development 
activity that the author would not consider to be MIS. This 
has had a confounding effect because the requirements for 
effective management of MIS activities appear to differ in 
some respects from the requirements for success with non-MIS 
activities. Specifically, it is the author's contention, 
supported by the data collected and analyzed in this thesis, 
that development of information systems for decision-making in 
the various functional and executive areas places a greater 
premium on involvement and integration of functional managers 
and executives than does development of non-MIS computer 
applications, which may have evolved fairly directly from 
existing procedures. This view is supported by several authors 
in the field (Canning & Sisson, 1967; Ditri & Wood, 1969; 
Lipperman, 1968; Reichenbach & Tasso, 1968). 

The last point above has particular relevance when 
viewed in the context of the future prospects for computer usage 


in organizations. Many organizations have about come to the 





a emma Ve Oe me — 
— 
om oii ed 
a 


1G 


end of the road on noneMIS computer applications, and are, 
therefore, looking to the opportunities offered by the more 
sophisticated and complex MIS applications. A report by 
Scientific American, entitled "The Computer Market" (1968), 
showed that only 39.9% of 1450 responding organizations were 
using computers for MIS applications at the time of the study, 
compared to 73% using computers for accounting and payroll 
applications. However, the report showed that 90% of the 
responding organizations planned to use computers for MIS 
applications in the future (p. 17). Thus, if there are major 
differences in the organizational and procedural variables 
related to success with MIS projects, as opposed to those 
variables related to success with noneMIS projects, findings of 
previous studies of the total organization's MIS efforts may 
not hold up in this new MIS environment. On the other hand, 

it may be that the same variables work in the same directions 
in both cases, or that the findings of earlier studies may have 
reflected the MIS part of the application mix. Unfortunately, 
the resolution of these uncertainties is impossible due to the 
mature of the data reported in previous studies. 

Finally, the research reported in this thesis is a 
break from the tradition of the intensive one-case study at 
one extreme, and the large-scale questionnaire survey at the 
other. The approach in this research has followed the lines 


proposed by Mouzelis (1969), in that a small sample has been 
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studied rather intensively; not as intensively as for a one- 
case study, but more intensively than is possible with a wide- 
spread questionnaire survey. Further, the sample, although 
small at twenty projects from ten organizations, represents 

a diversity of organizational environments, which affords the 


Opportunity for greater generalization than the one-case approach. 


Organization of the Study 


The author's prime objective in conducting the 
research reported in this thesis has been to add to the body 
of knowledge concerning the development and implementation of 
MIS projects. Specifically, an effort has been made to 
determine what organizational and procedural factors relate to 
MIS project success. It was recognized at the outset that one 
research effort could only scratch the surface in a field where 
so little definitive research has been done. However, it was 
felt that a well structured exploratory study, dealing with 
empirical data, could provide not only answers to some perplex- 
ing questions in its own right, but, more importantly, help to 
define more explicitly those finer-grain questions upon which 
additional research could be focused. 

In pursuing the above objectives, the literature on 
information systems, computer data-processing, and project 
management was reviewed to determine what other authors have 
had to say that was relevant to the subject at hand. The 


results of that review are reported in Chapter II. 
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Based upon the literature survey, the author's own 
experience in the information systems field, and the suggestions 
of faculty members and fellow graduate students at the University 
of Minnesota, a research design was gradually evolved which it 
was believed would facilitate satisfying the research objectives. 
Essentially, the approach involved defining a number of hypotheses 
which could be tested with empirical data. These hypotheses, 
along with the criteria of MIS project success, are presented 
in Chapter III. 

Following the enumeration of hypotheses to be tested, 
and the specification of success criteria, the detailed 
methodology of the study was worked out. The original set of 
hypotheses was reduced to more manageable proportions; an 
interview instrument and procedures for data collection were 
developed and tested; the study sample was selected; and the 
method of data analysis was decided upon. Chapter IV deals with 
these aspects of the study methodology. 

The data that were collected have been presented in 
two ways: Chapter V deals with the organization and project 
samples in descriptive terms, which gives the reader a "feel" 
for the subjects included in the study; Chapters VI and VII 
present the results of statistical tests on the relationships 
among the hypothesis, criterion, and other variables for 
which data were collected. (See Appendix E for the list of 


all variables in the study.) 
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Chapter VIII consists of a summary of the study 
findings, along with some interpretive comments by the author. 
Conclusions drawn by the author, as well as suggestions for 


future research directions, are presented in Chapter IX. 
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CHAPTER II 


SURVEY OF RELEVANT LITERATURE 


The survey of literature relevant to the procedural 
and organizational variables relating to success with MIS 
projects will be developed along two basic lines: that 
literature essentially prescriptive in nature but not founded 
on empirical studies, and that literature reporting the 


findings of empirical studies. 


Prescriptive Literature 


The prescriptive literature will be considered in 
two different categories. First to be dealt with will be the 
literature pertaining to what can generally be classified as 
organizational factors. This category includes such subjects 
as the location of the MIS function within the overall organi- 
zation structure, the degree of executive involvement in MIS 
development, the participation of user departments in MIS 
development, the selection and training of MIS staff, and the 
specific organizational structure for given projects. 

The second category of prescriptive literature is 
concerned with MIS project planning and control factors, and 
is comprised of such topics as definition of project objectives, 
documentation, allocation of manpower resources to projects, 
cost estimating of projects, the development and application 


of standards to project personnel, and the use of various 
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project control techniques similar to CPM or PERT. 


Organizational Factors 


A topic that has received a great deal of attention 
in the literature on information systems is the location in the 
organization of the information systems function. Diebold (1964) 
has contended that organizations have failed to adapt to the 
requirements for effective information systems by not giving 
adequate attention to a specific place in the organization 
for MIS functions. Reichenbach and Tasso (1968) have pointed 
out that the location of the MIS function has been determined 
in many cases by the nature of early computer applications, 
not the requirements of the current situation. Since early 
use of the computer was often oriented to accounting functions, 
the computer activity was placed under the controller and has 
frequently been left there. Whitmore (1966) has argued that 
the location of the MIS function within the controller's 
organization is a natural placement, in consonance with the 
controller's overall responsibilities, and should, therefore, 
remain there. Others, notably Cole (1966) and Brandon (1970), 
have merely held that the MIS function should be placed at a 
high level in the organization, commensurate with the responsi- 
bilities of the MIS manager. A counter point of view to 
Whitmore's, expressed by Canning and Sisson (1967), and 


Aukerman (1966), is that the MIS function should definitely 
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not be under the controller or a financial executive; rather, 
that it should be located directly under the top executive as 
an independent function, Finally, Reif (1968) has reviewed 
the literature bearing on organizational placement of the MIS 
function and has provided an analysis of the different points 
of view. In summary, it would appear that the literature on 
organizational location of the MIS function leans toward the 
prescription of independent function status at a high organi- 
zational level, although this position is not unanimously held. 
A second organizational factor which has received 
attention in the literature is the necessity for top management 
and user interest and participation in directing and controlling 
the development of MIS (Aukerman, 1966; Brandon, 1970; Canning 
& Sisson, 1967; Gallagher, 1961). The basic position held in 
common among these writers is that only through active partici- 
pation and assumption of responsibility for results can execu- 
tives and managers expect to derive the potential benefits 
from their management information systems. In the absence of 
such positive direction, the MIS will end up being what the 
MIS staff provides, which may or may not be what is really 
needed (probably the latter). 
With respect to staffing the MIS function, Canning 
(1967) and Whitmore (1966), among others, have prescribed 
means of selecting, developing and organizing the MIS staff. 


Campise (1968) has paid special attention to the need for 
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highly qualified MIS project leaders who are capable of 
effective communication with user personnel. Finally, Canning 
and Sisson (1967), and Ditri and Wood (1969) have advocated 
the use of interfunctional project teams composed of MIS staff 
and user personnel as a means of achieving higher quality MIS 


products and user organization acceptance of change. 


Project Planning and Control Factors 


Various aspects of MIS project planning and control 
have been dealt with in the literature. Probably the best known 
sources are Lecht (1967) and Brandon (1963; 1968). Lecht covers 
virtually the entire range of activities relating to project 
planning and control, while Brandon's scope has been more 
narrowly confined to the development and application of perform- 
ance and methods standards. LaBolle, et al (1965), working at 
the System Development Corporation, have developed a comprehen- 
sive planning guide for computer project development which also 
deals with almost all areas of project planning and control. 

Schwartz (1964) has stated that ''Inadequate planning 
is at the root of most data-processing failures and over-runs 
(p. 14)."" He has proposed using a "Work Breakdown Structure" 
to formalize and improve the planning, direction, and control 
of projects. Mensh (1969) has advocated using techniques which 
appear to be quite close to Schwartz's work breakdown structure. 

Blumenthal (1969) has provided what is probably the 


most thorough and articulate statement of the way in which an 
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organization should approach its total MIS effort, as well as 
how specific projects should be handled. Blumenthal advocates 
setting up an analytical framework based on functional areas 
of the organization. All proposals for MIS projects are then 
evaluated within the context of this framework. Blumenthal 
also provides a detailed, step by step procedure for the 
generation, selection, development, implementation, and audit 
of MIS projects. 

Some authors have recommended the use of such 
formalized project planning and control techniques as PERT 
(Canning & Sisson, 1967; Tannert, 1969) and PEST (Program 
Planning and Scheduling Technique, Anderson, 1966). The 
difficulty usually cited as inhibiting the utility of these 
critical path methods is the inability to make accurate 
resource requirement and time estimates. Pietrasanta (1968) 
has dealt explicitly with this problem in a discussion of 
estimates for manpower, machine-time, money, and project time. 
A much more thorough treatment of this same problem, and one 
based on empirical data collected by questionnaires for 169 
computer programs, has been provided by Nelson (1967). 

Finally, documentation methods and requirements for 
MIS projects have been proposed by Snyder (1965), Kay (1969), 
and Tannert (1969). In fact, with regard to project documenta- 
tion, nearly every writer in the field of MIS management has 
lamented the pervasive lack of adequate project documentation, 


and exhorted MIS management to recognize and deal with this problem. 
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Empirical Studies 


As was indicated earlier, all of the empirical 
research which has been reported (or at least found by the 
author) in the MIS area has focused on the overall organization 
rather than the individual project (with the exception of the 
SDC work on programming costs; Nelson, 1967). Most of this 
research has been accomplished either by management consultants 
or the American Management Association. The data for these 
various studies have generally been collected by questionnaire 
or interview, in some cases a combination of the two. For ease 
of presentation, the empirical studies will be discussed in 
chronological order. 

Based on the experience of 124 companies with data 
processing systems, Baumes (1961) drew several conclusions which 
he put in the form of recommendations for any organization 
planning to develop a computerized information system, The key 
point made by Baumes was the need for detailed planning of the 
tasks involved in information systems development. Such planning 
should be predicated on a clear statement of organizational 
objectives, and should definitely give close attention to 
behavioral factors, as well as technical factors. 

Thurston (1962) has reported a study of 32 case 
histories of organizations’ data processing activities. His 
primary conclusion was that a critical element in success of 


information systems was operating management's control of systems 
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planning. 

Perhaps the best known of the several studies which 
have been conducted by management consultants were the two by 
McKinsey & Company (Garrity, 1963; Unlocking the Computer's 
Profit Potential, 1968). Both of these surveys, based entirely 
on interview data, were aimed at determining what organizational 
and methodological factors had the greatest bearing on the 
successful usage of computers in the organizations studied. The 
1963 survey consisted of a sample of 27 companies representing 
13 industries. The 1968 survey covered 36 companies, also 
representing 13 industries. Those generally interviewed were 
the chief executive officer, the head of data systems, and 
other selected members of management concerned with the problem 
of effective computer utilization. | 

The conclusions drawn from both McKinsey surveys 
were essentially the same. These conclusions, briefly, were 
the need for top-management involvement from systems conception 
through implementation; the need for expecting top performance 
from the systems group across a wide spectrum of applications; 
and the need for the participation of user personnel in all 
levels of systems design and implementation. Those companies 
surveyed which were judged successful in their computer 

lthis information was provided by Mr. David B. Hertz of 


McKinsey & Co. in personel correspondence with the author 
dated 26 November 1968. 
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operations all evidenced these organizational and managerial 
characteristics. 

A very general questionnaire survey of 300 business 
firms using computers was conducted in 1964 by Business Automation 
(Kornblum, 1964). The objective of this survey was to find out 
how successfully computer installations were being used and 
managed. It appeared that the questionnaires were completed 
mainly by data processing personnel as the conclusions seemed 
to reflect the attitudes of the computer systems staff toward 
others in the organization. For the most part, the opinions 
expressed indicated that those outside the computer systems 
function did not understand the computer or how to use it; they 
did not understand the unique problems and needs of computer 
systems personnel; and they resented the encroachments of computer 
systems in their functional areas. 

The American Management Association sponsored a 
survey in 1965 (Higginson, 1965) with the primary objective 
of determining how the companies studied were using computer 
equipment, and the impact of computer systems on their organiza- 
tions and operations. This survey was conducted both by question- 
naire and interview. Questionnaires were returned by 288 
companies of 966 solicited, and interviews were conducted in 
21 firms to get more inedepth information as well as to get a 
subjective "feel" for the attitudes of management personnel. 


The conclusions were quite similar to those already cited from 
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earlier studies: the need for greater top-management direction 
and control of the computer systems function; the need for 
closer integration of staff specialists and operational manage- 
ment in the development and evaluation of information systems; 
and the need to consider the computer systems function just like 
any other functional unit with respect to organizational 
structure and evaluation of effectiveness in terms of corporate 
objectives. 

Two comprehensive surveys have been conducted by 


Booz, Allen and Hamilton, Inc., the first in 1966, and the second 


a year later (Computer Operations in Manufacturing Companies, 


1966; Computer Management in Manufacturing Companies, 1967). 


Both surveys were virtually identical in purpose, varying 
primarily in the sample sizes and means of collecting data. 

In the first Booz, Allen & Hamilton survey, the 
stated purpose was "to determine what successful manufacturing 
companies have done, are doing, and plan to accomplish in 
using and managing the computer.'' The key here was "successful". 
Only those companies in the various industries studied with the 
fastest growth and greatest return on investment were surveyed, 
since it was the objective to determine how exceptionally well 
managed companies were using the computer. The second survey, 
conducted in 1967, was different only to the extent that a wider 


range of data covering an expanded sample was sought. 
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The 1966 survey was conducted entirely by interview, 
using no questionnaire as far as can be ascertained. Thirty- 
three companies were included in the survey, and 189 interviews 
were conducted within them. Over half of the inedepth inter- 
views were conducted with top and operating management. The 
findings were then evaluated in an attempt to correlate computer 
usage and management with success of the company in question. 
The 1967 survey used a sample of 108 manufacturing companies, 
and questionnaires were employed to collect the data. The 
questionnaires covered organizational structure and practices 
in each company, as well as specific information on computer 
costs, usage and plans. 

The conclusions drawn from both surveys were mainly 
enumerations of the various composite computer management 
characteristics for successful companies. The reports did 
indicate that a trend was developing toward more top management 
and operational management involvement in computer systems plans 
and audits. Further, it appeared that the trend was to move 
the overall responsibility for computer activities away from 
financial executives to a computer executive at top corporate 
level. 

A questionnaire survey, conducted in 1967 as part of 
the Diebold Research Program (Summary Report of a Survey on 
the Cost Effectiveness of Software & Hardware, 1967), was 


designed to explore the future management implications of 
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computer information systems, and to develop some standard 
measurements of the cost effectiveness of hardware and software. 
The respondents to the questionnaire represented 17% of those 
solicited, and consisted of senior information systems manage- 
ment, technical staff reporting to information systems manage- 
ment, and top corporate management. The findings of relevance 
here were that in the majority of cases top management was not 
active in guiding the growth of information systems activities, 
and that new computer applications were often proposed by 
information systems management rather than those closer to the 
application area, namely, user management. The major problem 
with proper utilization of computer information systems was 
found to be the lack of concern on the part of management for 
maximum utilization. It was also concluded that larger computer 
users were more likely to have budget and control procedures 

for information systems activities, and were generally getting 
greater return on their information systems investments than 
smaller users. 

Lipperman (1968) has reported the results of investi- 
gations in twelve organizations using computer information 
systems. His data, collected by interview, were organized on 
a case basis to portray what various types of organizations 
have done to successfully implement computer systems. Lipperman 
dealt with both "operative" systems and planning systems, and 


pointed to the latter as the direction in which information 
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systems should develop as operative subsystems are integrated 
at higher levels. This particular study would seem to be the 
first dealing specifically with MIS as defined in this thesis. 

Several points were made by Lipperman which are of 
special interest here. As organizations moved toward higher 
level systems that integrated information from various subsystems, 
more user management responsibility for defining system require- 
ments was essential. Following from this was the need to 
locate the information systems function high in the management 
hierarchy, independent of such other functional executives as 
the controller or financial vice-president. 

A study undertaken for the American Management Associa- 
tion, under the direction of Reichenbach and Tasso (1968), 
was based on the hypothesis that "the location of the computer 
complex within the organization will affect the complex's opera- 
tions (p. 16)."" This study consisted of in-depth interviews 
with three levels of managememt in sixteen companies representing 
nine industries. The organizational members interviewed were 
top executives, information systems personnel, and user organi- 
zation managers. Although numerous conclusions were drawn 
from the sixteen case studies, the gist of the findings can be 
summarized briefly as follows: 

The location of the computer complex does have a 
very real relationship with its effectiveness in 


meeting organization needs. (No evidence was 
provided on the measure or meaning of effectiveness. ) 
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The computer information systems function 
should report directly to top management, 
and should not be a part of any existing 

traditional function. 


The centralization of the computer activi- 
ties need not have any impact on the 
traditional form of the corporate structure. 
The latter is a function of management's 
philosophy and objectives. 


The nature of an organization's basic 
activities and processes bears on the 

ease of its development of computer infor- 
mation systems. 


Participation by top level and user manage- 
ment in the development of information 
systems will enhance acceptance of those 
systems, improve utilization of the compu- 
ter, and provide results more responsive 

to management's needs. 


A monograph by Reif (1968) reported the findings of 
research he conducted for his doctoral dissertation in 1966. 
The literature survey comprising Chapter 2 of this monograph 
has been cited previously. The focus here will be on Reif's 
research methodology and findings. The impetus behind Reif's 
research was his belief that a major obstacle facing organiza- 
tions attempting to achieve maximum benefits from computer 
Systems was the conflict between the imperatives of computer 
technology and the traditional structure of business organizations. 

"There is a growing number of well-documented 

studies which show that the two are not com- 

patible; yet most firms today are content to 

achieve limited results by super-imposing 

advanced information systems on organizational 

structures which are unable to adjust to the 

new requirements and interpersonal relation- 


ships dictated by the systems design (from 
the Preface, p. iv)." 
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The specific purpose of Reif's study was "to examine what 
structural changes occur following the implementation of 
computer systems in business organizations (p. 41)."' This 
purpose was enumerated into several explicit factors such as 
the locus of decision making within the management hierarchy, 
formal and informal communication channels, and the functional 
integration of organizational activities. In addition, Reif 
studied the role of the computer systems staff, its location 
in the organization, and its relationships with user departments. 

Reif used the case method for studying three firms: 
a utility company, a bank, and a manufacturing firm. Most of 
the data were collected by interviews, both structured and 
unstructured, The remainder of the data came from company 
records. Before considering Reif's findings, it is important 
to note that his data reflect a considerable number of non-MIS 
computer applications, what might be termed "production" jobs, 
as opposed to the kind of decision oriented management informa- 
tion which is the focus of the current research, 

Based on his research, some of Reif's conclusions 
were: 

The presence of the computer resulted in 

centralizing decision-making within the 

management hierarchy. 

Linee-staff relationships were changed. 

The computer staff group exerted great 

influence and made decisions affecting 

the internal activities of operating 

departments. This staff influence was 


de facto, not usually legitimized by 
formal organization structure. 
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Lateral flows of communication throughout 
the management hierarchy were formed and 
expanded. However, formal channels of 

communication were not generally revised. 


The impact of computer systems will be 

greatest on middle management as routine 

administration and control decisions 

become programmed; these positions will 

generally decrease in number. However, 

the impact on middle managers will vary, 

depending on the organization, top 

management's objectives, and the 

functional area concerned. 

The hierarchical structure will be 

retained in organizations but "certain 

changes in the management framework 

appear to be imperative if the firm 

is to maintain harmony between infor- 

mation technology and the structure 

of organization (p. 114)." 

In an attempt to measure the effectiveness of manage- 
ment information systems in terms of profitability, a questionnaire 


survey was initiated by S. D. Leidesdorf & Co. in 1967 (The 
Profitability of Management Information Systems, 1969). Of 142 
companies requested to participate in this survey, 130 did so. 
These 130 companies, representing eleven industry groupings, 
were known to be using computers, "presumably effectively." 
Although the objective of the Leidesdorf study dealt 
specifically with the terms MIS and profitability, the use of 
these terms may be somewhat misleading. The profitability 
measure was derived by asking the participating companies ''to 
express as a percentage the influence of their computer operations 
and associated expenses on profitability during their first five 


years (or less, where that was appropriate) of computer-based 
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data processing (p. 18).'' This approach raises two issues: 
first, from information contained in the study report, it is 
impossible to determine what data went into each company's 
response on profitability, or how that data was aggregated 
to make the requested computation; and second, since the 
requested computation dealt with the early period of each 
company's computer use, it is highly probable that return from 
MIS applications was greatly confounded, if not overshadowed, 
by return from none-MIS applications. This latter observation 
is made in view of the tendency for organizations installing 
computers to implement costedisplacement applications first, 
with MIS applications following as more experience and sophis- 
tication are acquired. 

With respect to the results of the Leidesdorf study, 
32 of the 130 companies were viewed as being successful with 
their MIS in terms of the reported percentage profitability 
contribution of MIS. The study concluded that these 32 
companies achieved the high MIS contribution to profitability 
by "intelligent planning and control". For the 98 companies 
viewed as not successful in terms of MIS contribution to 
profitability, the report stated that: 

"The common denominator seems to be defi- 

ciencies in planning associated with the 

operation. To a lesser extent, the absence 

or incompetence of functional and/or execu- 

tive participation in the planning and 


operation of the project appears to be a 
contributing factor (p. 21)." 
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To summarize the Leidesdorf study, it was concluded 
that: 

"Management information systems represent 

a favorable field for investment and can 

make a real contribution to profitability-- 

especially through improved control of the 

business; but positive results are not 

automatic and guaranteed. Serious problems 

can and do arise and are usually attribut- 

able to deficiencies in management involve- 

ment, planning, and control (p. 24)." 

Before concluding this section on the survey of the 
literature, two doctoral dissertations warrant mention. Holsinger 
(1970) studied the constraints to implementation of data process=- 
ing systems, and examined the roles of key decision makers 
involved in implementing such systems. Holsinger obtained his 
data through interviews with members of top management, key 
technical personnel, and ''third party representatives.'' The 
constraining variables were categorized as technological, organi- 
zational, and individual. Holsinger concluded that the relative 
importance of these three categories has shifted over time; 
that as size increases, the importance of technology and organi- 
zation increases at the same time that the importance of the 
individual decreases; that size influences the organization's 
ability to utilize a proper division of labor; that control 
problems were created through the determination by technologists 
of critical aspects of systems design; and that structural 


problems of integrating systems functions with various types of 


information systems appeared. 
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The second dissertation, by Hcdgetts (1968), is of 
interest since it deals with project management. Based on the 
research he conducted, Hodgetts advocates the project approach 
as opposed to the functional approach where an organization is 
confronted with managing a specific, complex task with stringent 
quality parameters. He enumerates the pros and cons of the 
project management approach and points out that it is a manage- 
ment tool not an organizational panacea. Of particular interest 
are his observations concerning the interface problems between 
the project organization and the permanent organization, and 
the attributes of effective project managers. Although Hodgetts 
was concerned primarily with project management in the aero- 
space, construction, chemical, and consumer products industries, 
and not with MIS projects, his inter-industry approach, the 
focus on individual projects and the evaluation of project tech- 


niques and relationships are relevant to the present research. 


summary 


Although the literature relevant to management infor- 
mation systems is rather extensive, there are few reports of 
well conceived and structured research in the field. While 
prescriptions and definitions have been plentiful, they have 
seldom been supported by data. Numerous authors have presented 
their views on virtually all aspects of information systems, 
but these views have generally stemmed from their authors' 


personal experiences rather than structured research. Those 
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research studies which have been reported have tended to be 
broad-brush. They have generally represented wide-scale 
questionnaire and interview surveys which were aimed at overall 
organizational use of computers, or case studies of the impact 
of computer usage on organization structures and relationships. 

The above comments concerning the literature relevant 
to MIS are not intended as criticism of what others have done. 
Rather, the point to be made is that in a field growing so 
rapidly, with so few definitive guidelines, a great deal more 
needs to be done. Researchers and practitioners in the MIS 
field must define areas to be researched, then develop well- 
conceived approaches to conducting such research. What has 
already been accomplished will be helpful, as, indeed, it has 
been helpful to the author in conceiving and designing the 
research reported in this thesis. But the overwhelming impression 
one is left with after reviewing the information systems litera- 
ture is the great gap between what is needed and what has been 
accomplished. 

Building on what others have said and done, the 
author has made an attempt to start closing that gap. The 
prescriptions and empirical findings of others were major 
contributions to the definition of the hypotheses posited and 
tested in the present research. The techniques others have 
employed have been invaluable guides to the author in designing 


the data collection methodology used in this research. 
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Nonetheless, the research conducted by the author, and reported 
in this thesis, represents a departure from previous research 

in both approach and substance. Rather than collecting a mass 
of data, then setting up hypotheses to be tested, the author 

of the present study set up the hypotheses, then developed data 
collection and analysis techniques to test them. As opposed 

to looking at overall computer usage in many organizations, or 
the intensive study of one or two organizations to determine 
what impact computer usage has had on them, the present research 
has focused on the MIS project. Specific, measurable criteria 
of project success have been defined in order that the hypotheses 


posited could be tested. 









—* 
—a— a om 
oe oo 
_— 
— «—§ 0 =e —_ 
—— a) 
ee @ = 
SA ee ym 
- = = 
Se et ed 
eee es oe es 
OE a OE me mm 
et ee Seti + 
eet ee ee *) oe 
ee ae -4, ee <— — — 
est wae <1 e © 





























=95¢ 


Nonetheless, the research conducted by the author, and reported 
in this thesis, represents a departure from previous research 

in both approach and substance. Rather than collecting a mass 
of data, then setting up hypotheses to be tested, the author 

of the present study set up the hypotheses, then developed data 
collection and analysis techniques to test them. As opposed 

to looking at overall computer usage in many organizations, or 
the intensive study of one or two organizations to determine 
what impact computer usage has had on them, the present research 
has focused on the MIS project. Specific, measurable criteria 
of project success have been defined in order that the hypotheses 


posited could be tested. 





PART II 


HYPOTHESES, CRITERIA OF SUCCESS, AND 


METHODOLOGY OF THE STUDY 


This part of the thesis deals with the means by which it was 
attempted to formulate answers to the question posed earlier: 


“What organizational and procedural factors 
are correlates of success with MIS projects?" 
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CHAPTER III 


HYPOTHESES AND CRITERIA OF SUCCESS 


Introduction 

To assist in identifying the organizational and 
procedural factors related to success with MIS projects, a 
list of thirty-five hypotheses was developed. These hypotheses 
were constructed from the literature previously reviewed, from 
the author's personal experience, and from discussions with 
various professionals in the MIS field. The hypotheses provided 
a framework for the collection and analysis of data relevant 
to the purpose stated above. 

In addition to the list of hypotheses, a set of 
criteria was developed to evaluate relative success of given 
MIS projects. The criteria will be taken up first, followed 


by a discussion of the hypotheses. 


Criteria of Success 

A given MIS project can be viewed from several 
perspectives in attempting to judge whether or not it was 
successful, It is necessary that the criteria selected be 
relevant, be as objective as possible, and at the same time be 
realistic in terms of ease and consistency of measurement. Four 
separate criteria which appear to meet these requirements are: 

Ll, Success in terms of time. How closely 

did actual time spent on the project 


conform to the time allocated for it? 
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Success in terms of development cost. 

Was the project completed within the 
budgeted cost? If over-runs were 
experienced, what was the total cost 

as a percentage of budgeted cost? 

Success in terms of users' evaluations. 
Did the completed project satisfy user 
requirements and expectations? Were 

there unanticipated problems or benefits 
resulting from implementing the completed 
project? 

Success in terms of computer operations. 
Has the completed project created problems 
for computer operations in scheduling, set 


up, run time, or control? 


It is believed that the above criteria are relevant 


operational measures of project success, and, as such, provided 


a suitable means of testing the hypotheses selected for evaluation. 


Hypotheses 


The thirty-five factors which were hypothesized to be 


categories: 


| 


2 


related to MIS project success fall into the following broad 


Characteristics and competence of project 
personnel. 


Project management and control techniques. 
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3. Organizational interaction factors. 

4. Project specific factors. 

5. Global factors comprising the project 

environment. 
The hypotheses were not viewed as exhaustive nor were their 
assignments to categories clear cut in all cases. However, it 
was felt that both the hypotheses, and the categories into which 
they were placed, provided a suitable framework for conducting 
research. 

The hypotheses are presented below by categories, 

with a brief description of what each category represents. The 
individual factors or hypotheses should be self-explanatory. 
In order to avoid redundancy in the list of hypotheses, an 
example of the form for each full hypothesis will be given 
here. The list itself will merely consist of the body of the 
given hypothesis in the form of a factor ostensibly related to 
MIS project success. 

Example: It is hypothesized that the higher 

the level of formal education of 

project personnel, the greater the 

likelihood of MIS project success. 
Characteristics and C etence of Project Personnel (Category I 

This category should require little explanation. It 
is merely a grouping of those hypotheses about the competence 
of people who worked directly on the project. While the general 


statement ''the more competent the personnel working on the project, 
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the more likely success" is something of a truism, the factors 


falling under competence may be debatable in certain cases. 


Such factors 


are, then, testable. 


Coordinating ability of project leader. 
Systems experience of project personnel. 
Persuasiveness of project leader (as 
evaluated by superiors). 

Proficiency of project personnel (as 
judged by superiors). 

Low turnover of project personnel. 
Length of experience in the organiza- 
tion of project personnel. 

High formal educational level of 


project personnel. 


Organizational Interaction Factors (Category II 


This category includes those hypotheses dealing with 


how the user organization interfaces with, supports, and is 


supported by the MIS department. It is, in short, a group of 


hypotheses focused on the integration of the overall organiza- 


tion with respect to development and utilization of management 


information systems, 


8. 


Participation by operating management in 
design, formal approval of specifications, 


and continual review of project. 





10. 
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Utilization of a project team composed 
of MIS staff and user personnel. 
Operating management conducts periodic 
management audit of MIS function. 
(Evaluation of effectiveness for users). 
Organizational level of top computer 
executive. 

Formal training program set up for 

user organization. 


Project originated by user organization. 


Project Management and Control Techniques (Category III 


Subsumed within this category are those hypotheses 


pertaining to specific methods used to plan, direct, monitor, 


and control a given project. This group represents many of 


what are frequently prescribed as "good practices" in the 


literature on management of the data processing function. 


14, 
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18. 


Measurable project objectives from 
conception of the project. 

Formal project selection process used 

to determine which projects to develop. 
Documentation standards used and enforced, 
Use of a formalized and regular reporting 
structure on project progress. 

Planning and accounting for all resources 


throughout project development. 
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19. Program maintenance and review responsibility 
specified for definite period after implementation. 
20. Utilization of a formal time-scheduling 
technique such as PERT for project development. 
21. Performance standards employed for analysts 


and programmers. 


Project Specific Factors (Category IV 
This category encompasses the various factors that 
are unique to a given MIS project, such as the programming 
language used. 
22. High availability of computer time for 
program testing. 
23. High level programming language used 
for project. 
24. Utilize existing data base versus 
constructing or greatly modifying one. 
25. Short-term, minor project versus large, 


complex project. 


Global Factors Making up the Project Environment (Category V 
The global category covers a rather wide range of 
factors as the name would imply. These are environmental 
variables which might be viewed as more indirect in effect than 
the factors under other classifications. In a sense this is a 


sort of omnibus category that includes the hypotheses that were 
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considered relevant but did not easily fit into a more specific 


grouping. 


26. 


27. 


28. 


Zo. 


30. 


31. 


i Ap 


38. 


34, 


Jo. 


This 


High centralization of organizational 

MIS activities. 

Number of years experience for organiza- 
tion with computerized information systems. 
Low turnover rate of MIS staff. 

Combination analyst/programmers for 

small projects. 

High rates of MIS staff drawn from 

within the organization, 

High average income level of MIS staff. 

Low degree of overall organizational 
change. 

Separation of analysts and programmers 

for large projects. 

Overall size of organization systems staff. 
Ratio of computer hardware investment to 


total sales or operating budget. 


summary 


chapter has been devoted to presenting the factors 


which were hypothesized to be related to MIS project success and 


to defining specific criteria of success suitable for testing 


those hypotheses. Chapter IV details the methods which were used 


to test the hypotheses. 
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CHAPTER IV 


METHODOLOGY OF THE STUDY 


Initial Evaluation of the Hypotheses 


The above list of thirty-five hypotheses would impose 
a considerable burden on anyone desiring to test them in field 
research, For this reason, it was felt advisable to try to 
sort out the various hypotheses into some kind of priority 
hierarchy. The result of such a sorting process would be a 
ranking of all the factors, from those viewed as most crucial 
to project success to those viewed as least significant. It 
was decided that the best means of coming up with such a 
priority ranking of the hypotheses was to get the opinions of 
professionals in the MIS field, 

The opportunity for getting the opinions of MIS 
professionals on the importance of the various hypotheses to 
project success was afforded by the Founding Conference of the 
Society for Management Information Systems (SMIS), held at the 
University of Minnesota in late 1969. The decision was made to 
submit the hypotheses in questionnaire form to the attendees of 
this conference. The expected product of the completed question- 


naires was to be the desired priority ranking. 


Pre-questionnaire 


To facilitate developing a questionnaire which, when 


completed, could be evaluated in such a way as to provide the 
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ranking by importance of the hypotheses, it was decided that a 
pre-questionnaire should be prepared and submitted to a group 
of MIS professionals in the Twin Cities area. The respondents 
(N=43) for the pre-questionnaire were all employed by the firms 
associated with the Management Information Systems Research 
Center of the University of Minnesota. 

The objective in preparing and evaluating the pre- 
questionnaire was to come up with a "neutral" hypothesis which 
could be used as a reference or anchor point in constructing the 
final questionnaire to be submitted to the SMIS conference 
participants. This neutral hypothesis was to be the basis of 
comparison for each of the remaining thirty-four hypotheses. 

By choosing a "neutral" hypothesis as the standard for comparison, 
one seldom viewed as either most important or least important 

to MIS project success by the pre-questionnaire respondents, 

it was expected that the desired spread in priority rankings 

would be derived from the final questionnaire. 

The procedure employed for the pre-questionnaire was 
to list the thirty-five hypotheses, or factors, disregarding 
any classification by categories. Opposite each hypothesis 
were two blanks, one under a column headed "Most Important"! 
and the other under a column headed "Least Important''. The 
respondent was then requested to evaluate each of the thirty- 
five factors as to its criticality for MIS project success, 


and to indicate by the numbers 1 to 7 those seven factors 
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considered to be most important, and those seven factors 
considered least important, in contributing to project success. 


(See Figure 1). 


FIGURE 1 


Pre-ques tionnaire 


Most Least 
Factor Important Important 
1) High formal educational level 
of project personnel 
2) Proficiency of project personnel 
as judged by superiors 
35) Etc. Etc, Etc. 


The returned pre-questionnaires were analyzed to 
determine which factor was "most neutral". That is, which 
factor was regarded to be of middle importance in the range of 
thirty-five. This factor was, of course, one that was seldom 
ranked as one of the seven most or seven least important in 


contributing to MIS project success. 


SMIS Questionnaire 
Analysis of the responses showed the "most neutral" 
factor to be: ‘Performance standards employed for analysts 


and programmers". This factor was then used to construct the 
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final questionnaire in the following manner. The remaining 
thirty-four factors were listed as before with no categorization. 
Opposite each factor was a Likertetype scale allowing for one 
of five responses to each item. At the top of each page of the 
questionnaire the neutral (standard) factor was printed. The 
respondent was then asked to rate each of the thirty-four 
remaining factors for importance in contributing to MIS project 
success by comparing it to the standard factor. The form of 
the response was merely a check in the chosen one of the five 
blanks. In addition, each respondent was provided space at the 
end of the questionnaire to write in any other factors which 
he believed important to MIS project success, but which were 
not included in the list of thirty-four factors to be rated 
(see Appendix A). 
Of the roughly 250 participants in the SMIS conference, 
142 returned the questionnaire. The returned questionnaires 
were tallied by determining the number of responses in each of 
the five levels for each factor. Each level was then given a 
numerical value as follows: 
Much less 
Somewhat less 
About same 


Somewhat more 
Much more 
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Multiplying the number of responses per level by the 
level value, summing across all levels for a factor, then 


dividing by the number of responses for that factor, yielded a 
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numerical score for each factor. The factors were then ordered 
on the basis of these scores to give a ranking from most 


important to least important. 


Results from the SMIS Questionnaire 


Evaluation of the returned questionnaires resulted in 
the factor ranking shown in Figure 2, The scores for the 
individual factors ranged from 4.38 for the factor judged most 
important for MIS project success to 2.30 for the least important 
factor. In addition, as will be noted in Figure 2, the variance 
was calculated for each factor. These variances, ranging from 
.602 for the factor ranked second in importance to 1.519 for 
the factor ranked seventeenth, were computed to determine the 
relative agreement among respondents on the factors, and were 


not used for any purpose beyond that. 


Analysis of SMIS Questionnaire Data 


As was pointed out earlier, the questionnaire was 
constructed and administered to facilitate field study of the 
hypotheses. The ranking provided a means of focusing on what 
factors those active in the MIS field believed to be the most, 
the least, and of intermediate importance to project success. 
The next steps were to select the hypotheses to be further 
evaluated, and to design a means of collecting data in organi- 


zations on both the hypotheses and the criteria. 
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FIGURE 2 


Ranking of Hypotheses 


Computed 
score 

Participation by operating management ( wx /N) 
in design, formal approval of specifi- i 
cations and continual review of 
project. 4.38 
Measurable project objectives from 
conception of the project. 4.29 
Utilization of a project team composed 
of MIS staff and user personnel. 4,25 
Coordinating ability of project leader. 4.21 


Operating management conducts periodic 
management audit of MIS function. 
(Evaluation of effectiveness for users). 


Formal training program set up for 
user organization. 


Organizational level of top computer 
executive. 


Systems experience of project personnel. 


Formal project selection process used 
to determine which projects to develop. 


Persuasiveness of project leader 
(superior's evaluation). 


Proficiency of project personnel 
(as judged by superiors). 


Documentation standards used and 
enforced. 


Use of a formalized and regular report- 
ing structure on project progress. 


Low turnover of project personnel. 


* Hypotheses selected for further study. 
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Planning and accounting for all 
resources throughout project develop- 
ment. 


Source of origination of project (MIS 
staff or user). 


High centralization or organizational 
MIS activities. 


Program maintenance and review 
responsibility specified for definite 
period after implementation. 


Number of years experience for 
organization with computerized 
information systems. 


Length of experience in the organiza- 
tion of project personnel. 


Utilization of a formal time-scheduling 
technique such as PERT for project 
development. 


High availability of computer time for 
program testing. 


High level programming language used 
for project. 


Utilize existing data base versus 
constructing or greatly modifying one. 


Low turnover rate of MIS staff. 


Short-term, minor project versus 
large, complex project. 


Combination analyst/programmer for 
small projects. 


* Hypotheses selected for further study. 
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Computed 


Score Variance 


High rates of MIS staff drawn from 
within the organization. 2e01 1.104 


High average income level of MIS staff. Zete ~750 


Low degree of overall organizational 
change. Lad 2 1.263 


High formal educational level of 
project personnel. 2.71 1,109 


Separation of analysts and programmers 
for large projects. Zio 1.274 


Overall size of organization systems 
staff. 2.50 1.207 


Ratio of computer hardware investment 
to total sales or operating budget, 2.30 1.188 


* Hypotheses selected for further study. 
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Two approaches were used in selecting the hypotheses 
for further evaluation. First, it was desired that about three 
factors be evaluated from the top, middle, and bottom of the 
ranked listing. Based on this, and the relative ease of 
collecting data on each of the factors, the following factors 
were tentatively selected for evaluation (numbers refer to 


ranks in Figure 2): 


Top pee me’ 
Middle 14, 16, 17 
Bottom 31, 32, 33, 34 


Second, a factor analysis was performed on the 
questionnaire data using the principal components method 
(see Guilford, 1954; Harman, 1965). The reason for this was 
to determine if items on the questionnaire represented certain 
basic dimensions or underlying factors. If present, such 
underlying factors could then be used as a guide in selecting 
individual hypotheses from the ranked list for evaluation. 

The factor analysis yielded eleven factors 
accounting for 65.9% of the variance. However, most of these 
factors accounted for a small part of the overall variance, 
and/or only one or two questionnaire items loaded to any 
appreciable extent on a given factor. The factor accounting 


for the largest amount of the variance (9.475%) represented 
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essentially those items that were ranked high on the question- 
naire (items 1, 2, 3, 5, 6, and 9 in Figure 2). The factor 
accounting for the second largest amount of the variance 
(9.456%) represented essentially those questionnaire items 
ranked low (items 17, 28, 29, 31, 32, 33 and 34 in Figure 2). 
A third factor, accounting for 6.44% of the variance, had 
three items loading very high on it (items 13, 15 and 21 in 
Figure 2), and represented what could be called "project 
control", The other eight factors were judged not to be 
meaningful for the purposes here, either because they accounted 
for such a small part of the variance or because only one 

or two items loaded on them to any extent. 

It was apparent from the factor analysis that the 
high ranked and low ranked items, comprising the first two 
factors, were well represented in the preliminary list of 
ten hypotheses selected for further evaluation. However, 
the project control factor was not represented among those 
ten. Therefore, item 13 from the ranked list (Figure 2) was 
added to the ten hypotheses previously selected for further 
evaluation, 

Besides the above eleven hypotheses, five more 
were selected to be further evaluated. These were items 7, 

8, 12, 20, and 21 from the ranked list of hypotheses (Figure 2). 


The rationale for including these additional hypotheses was 
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(1) ease of collecting data concerning them; merely being 
in an organization would allow getting data on the hypotheses 
at virtually no extra cost, and (2) personal interest in the 
hypotheses because of related research currently going on 
or because of emphasis in the literature. 

As was noted earlier, space was provided for 
each respondent to write in any factors he felt important 
to MIS project success. All write-in responses were reviewed, 
but no new hypothesis was deemed necessary from this review. 
In most cases, the respondents merely restated in a different 
form one of the hypotheses in the list rated. In a few cases 
the write-in comments were not closely related to the 
hypotheses rated; however, the incidence of one or two comments 
on a subject was not viewed as justification for creating a 
new hypothesis. 

In summary, the result of the questionnaire 
analysis was the selection of those sixteen hypotheses identi- 
fied by an asterisk (*) in Figure 2 to be evaluated by field 


study. 


Data Collection in the Field 
It was believed that the best means of collecting 
data to test the selected hypotheses was through interviewing 


key persons involved in developing, or using the products of, 
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MIS projects in several organizations in several industries. 
This approach was to comprise a "descriptive' study, using the 
terms of Selltiz et al (1967). That is, the emphasis was on 
finding out if given variables were associated, in this case the 
hypotheses and the criteria of project success. 

With a view to the above objectives, a questionnaire 
to be used by the researcher in structured interviews was 
developed. This questionnaire (see Appendix B) was aimed at 
collecting several kinds of data and consisted of several parts. 
First, the kinds of data collected with the questionnaire can 
be described as independent variable data, dependent variable 
data, and moderator variable data. The independent variables 
were the hypotheses selected, as outlined above; the dependent 
variables were the criteria discussed earlier; and the moderator 
variables represented various organizational or procedural 
factors which might have influenced or moderated the relation- 
ships among the independent and dependent variables. 

The parts of the questionnaire reflect who was expected 
to answer certain kinds of questions about a project in the 
organization. Thus, certain questions about the organization 
itself were to be answered by the manager of the MIS function. 
The questions specifically related to the project were to be 
asked of the project leader, and the questions concerning user 


perceptions of participation and satisfaction with the products 
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of the project were to be directed to user management 
ae Senne. 
It was decided that twenty MIS projects, drawn from 
ten separate organizations, would constitute an adequate 
sample for the type of research to be conducted. This would 
represent two projects in each of the ten organizations. It 
was further decided that of the two projects in each organiza- 
tion, one of them should be viewed as relatively successful, 
and one relatively unsuccessful, by personnel in the informa- 
tion systems area of the organization. 
The general guidelines for selecting projects for 
study were specified as follows: 
1. Must be a MIS project as defined in Chapter I; 
not a data processing application with no 
MIS implications. 

2. At least two people worked full time in 
developing the project; the project required 
at least three elapsed months to develop. 

3. The project was completed and implemented 

within the last 6-24 months. 
The reason for guideline 2 above was to preclude studying 
very small projects which were completed by one person over a 


brief time, such as a simple report generation application from 


The following references were used in developing the 
questionnaire for conducting interviews: Cannell & Kahn, 
1953; Edwards & Kenney, 1946; Festinger & Katz, 1953; Payne, 
1951; Selltiz et al, 1967; Torgerson, 1953; Whitlock, 1963. 
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an existing file of data. The reasons for guideline 3 were 
to avoid projects which had been completed for so long that 
those involved were not available for interview, or, if 
available, their recollections were too fuzzy to be anywhere 
near accurate; and to avoid projects which had been completed 
so recently that there was inadequate user experience with 


the products to allow fair evaluation. 


Pretest of Interview Instrument 
It was next decided that a pretest of the questionnaire 
should be made prior to actually selecting a sample and making 
contacts for the full study. The objectives of the pretest 
were to determine if data of the types wanted could be gathered, 
and, if so, were the questionnaire and associated structured 
interview effective and efficient means to collect these data. 
Accordingly, to satisfy these objectives, and to 
obtain estimates of how long the data collection would take 
for each organization in the sample, a pretest of the question- 
naire and interview technique was conducted during the week of 
4 May 1970, in a Twin Cities financial institution. Based on 
the pretest, it was concluded that the data collection techniques 
planned were feasible, and would provide the data desired. 
However, certain very minor changes to the questionnaire appeared 
desirable as a result of the pretest experience. For the most 
part, the changes merely involved revisions of wording in 


questions which had caused some confusion among pretest 
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respondents. Two questions were dropped completely from the 
questionnaire as it was apparent they were redundant. These 

two questions elicited essentially the same information from 
respondents as two other questions which were left in the 
questionnaire. Three questions were added to the questionnaire, 
and two were modified substantially, in order that project 
leaders and users would be asked identical questions concerning 
user participation and implementation problems. None of these 
changes altered the substance of the questionnaire to any 
appreciable extent, and were, primarily, corrections for over- 
sights brought out by the pretest. After revising the question- 
naire, work began on selecting the organization sample to be 


used in the full field study. 


Organization Sample Selection 


As was stated at an earlier point, it was desired 
that the ten organizations comprising the study sample represent 
different industries to the greatest extent possible. This 
objective for selecting an organization sample was constrained 
by the fact that it was also desired that those organizations 
in the sample be Associates of the Management Information Systems 
Research Center (MISRC) at the University of Minnesota. 
However, this constraint was not a very serious one in that the 
Associate firms do represent a wide cross-section of industry 


types in the Twin Cities area. 
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With a view to the above requirements, a preliminary 
list of ten firms was prepared. Each of these ten firms was 
then sent a letter from the Associate Director of the MISRC, 
indicating the nature of the research to be conducted, the 
background of the researcher, and a request that the firm 
participate in the research (Appendix C). Within one to 
two weeks from the time the letter went out, the researcher 
contacted the Associate representative of each firm by tele- 
phone to request an appointment for the purpose of discussing 
the study and getting agreement from the firm to participate. 

During the last two weeks of July, 1970, each of 
the ten firms originally selected was visited by the researcher. 
These visits generally lasted about thirty minutes, during 
which time the researcher described what the study was to 
consist of, the kinds of data to be collected, and the types 
of people in the organization who would be involved if the 
firm agreed to participate. Additionally, each Associate 
representative with whom the researcher visited was given a 
brief written résume of the proposed research, which included 
the estimated duration of each interview to be conducted and 
the total time commitment by the organization (Appendix D). 

Of the initial ten organizations contacted, all but 
one agreed to participate in the study. The one exception was 
an organization which had begun exploring MIS applications 


very recently, and felt, therefore, that there would not be 
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any projects for study in the organization which staisfied the 
researcher's project selection specifications. However, to 
offset this organization's not participating in the study, one 
of the Associate representatives visited desired that two 
separate divisions of his organization be included in the 
study. The researcher agreed to this after satisfying himself 
that, for all practical purposes, the two highly autonomous 
divisions of this very large and diversified corporation were 
separate and independent with respect to the factors relevant 
to the study. 

Having gotten agreement to participate in the study 
from those organizations selected for the sample, a schedule 
was prepared for actual data collection. This schedule, covere- 
ing the period from early October to mid-December, 1970, allowed 
at least three days for collecting data in each organization. 
Each organization was again contacted by telephone to arrange 
the dates for interviewing in the organization. The only 
appointment made at this time was with the director of the 
information systems function or other individual(s) whom had 
been designated by the Associate representative to coordinate 


data collection in the organization. 


Data Collection Procedures 
The data collection procedures followed in each 


organization which participated in the study were as follows: 
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An initial meeting with the manager of the 
information systems function and/or his 
designated representatives was opened by 
explaining precisely what data were to 

be collected, and from whom, after a 

general description of the research study 
had been provided. Next, the organization 
representatives were asked to suggest 
projects for study which met the project 
selection specifications described earlier 

in this chapter. This proved to be a more 
time consuming task than expected, and, in 
some cases, required as long as three hours 
rather than the thirty minutes estimated 

from the pretest experience. Those two 
projects which best satisfied the selection 
specifications were chosen for study. The 
organization representatives were then 
requested by the researcher to arrange 
appointments with the project leaders of 

the two projects selected. Finally, the 
manager of information systems, or his repre- 
sentative, was asked to complete pages 1-3 of 
the questionnaire. This was the only part of 


the questionnaire which was not completed during 
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interviews with the researcher. However, on 
several occasions the researcher did assist 

the respondents for pages 1-3 by clarifying 
points that were confusing, or in making certain 
that the respondents interpreted the questions 
as the researcher intended. 

The meeting with the project leader was opened 
with a brief description of the nature of the 
study and the data to be collected, and with 
assurances to the project leader that all of 

his responses would be held in confidence by 

the researcher. The remainder of the 1% to 3 
hours spent with the project leader was devoted 
to completing the appropriate portions of the 
questionnaire. The respondent was given a 

copy of the questionnaire to read as the 
researcher asked the questions. 

Upon completion of his portion of the question- 
naire, the project leader was requested to 
arrange appointments with at least two managers 
in the area(s) which received the products of 
the project. For this purpose, a staff analyst 
or similar person was considered to be a manager. 
In general, if a user was involved in making 


planning and/or control decisions in a functional 
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area of responsibility, he was considered a 
manager whether he had any subordinates or not. 
Finally, the project leader was requested to 
arrange an appointment with the computer opera- 
tions manager, or other knowledgeable individual 
in the computer operations function who could 
answer those questions pertaining to computer 
operations for the project. 

The meeting with each user was opened in exactly 
the same manner as the meeting with the project 
leader. After the preliminary comments by the 
researcher about the study, the remainder of 
the fifteen to thirty minutes spent with the 
user was devoted to the completion of the 
relevant portions of the questionnaire. As 
with the project leader, the user was provided 
a blank questionnaire to read as the researcher 
asked the questions. 

The same preliminary explanations on the nature 
of the study, and so forth, were provided to 
the computer operations management. In most 
cases, the computer operations data for both 
projects in an organization were collected in 
one interview. This interview generally lasted 


from ten to fifteen minutes. 
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Steps 2 and 3 of the above procedure were repeated for the 


second project studied in the organization. 


Data Analysis Procedures 


After all the data had been collected in all ten 
organizations in the sample, the interview forms were reviewed 
and coded by the researcher. Since the questionnaire was 
originally developed to facilitate coding and conversion of 
the data to punched card form, this was a relatively easy task. 

Several methods of data analysis were considered, 
but it was decided that Kendall's (1962) tau statistic provided 
the best measure available of relationships among the variables 
for the following reasons: 

1. The tau statistic is robust and relatively 

powerful in dealing with ranked data contain- 
ing ties among ranks. 

2. Being a measure of relationship among ordinal 
variables, there are no assumptions concerning 
distribution of the raw data for the tau statistic. 

3. The S statistic, which is actually the numerator 
of the tau statistic, and is a measure of the 
agreement or disagreement between a pair of 
rankings, is distributed approximately normally 

anee discussions of the tau statistic, its distribution 
under various conditions, and corrections for continuity, see 


Burr, 1960; Goodman and Kruskal, 1954; Hoffding, 1947; Kendall, 
1947; 1962; Sillitto, 1947; Somers, 1962; and Whitfield, 1947. 
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for N>9. The S statistic can, therefore, 
be converted to a normal deviate,Z(s), after 
computing the variance of S. This normal 
deviate can then be tested, using the normal 
probability wie to ascertain the signi- 
ficance level of the relationship between 
two rankings. 

4. Since the data collected could be arranged 
in naturally ordered bivariate tables, the 
computation of tau, S, and Z(s) could be 
accomplished quite easily using the UMST 
(1969) programs available for the University 
of Minnesota's CDC 6600 computer system. 

The first step taken in analyzing the data collected 
was to prepare a descriptive profile of the sample. This 
descriptive profile was broken into two parts: descriptive 
statistics on the organizations in the sample, and descriptive 
statistics on the projects, These descriptive statistics 
consisted of means, medians, and ranges on those variables which 
were amenable to such treatment, and distributions of scores 
on the variables which were not. These descriptive statistics 
on the organizations and the projects are presented in Chapter 5. 

The second step in analyzing the data was to compute 
the tau statistics for relationships among the variable rankings. 


This step, in turn, was divided into three substeps. 





208. 


First, it was necessary to determine whether or not 
those variables which were originally intended to be composites 
of several separate items were, in fact, valid composites repre- 
senting the same factor, or were, in reality, separate factors 
which should not be aggregated. The variables in question 
were User Participation-PL (Var. 41), User Participation-User 
(Var. 46), User Satisfaction-User (Var. 54), project leader's 
perception of success, and operations success. To make this 
determination, Pearson product-moment correlation coefficients 
were computed for all of those separate items comprising a 
supposed aggregate factor. This computation was accomplished 
with computer program UMST 530. If any item within a supposed 
composite factor was not correlated at least .50 with every 
other item in that composite factor, that item was considered 
to be a separate measure and dropped from the composite factor. 
It was found that Variables 41, 46, and 54 did, in fact, have 
items which were intercorrelated at least .50, and, therefore, 
were acceptable as composite measures of the factors in 
question. However, the intercorrelations of separate items 
in the assumed composite variables representing the project 
leader's perception of success and computer operations success 
were so low as to preclude their being considered composite 
factors. Consequently, computer operations success and the 
project leader's perception of success were treated in all 


further analysis as two separate variables (59 and 60), and 
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four separate variables (55-58), respectively. (See Appendix E 
for descriptions of all variables used and their scoring.) 
Next, in order to determine roughly which variable 
rankings were related enough to warrant further investigation 
of the relationships, UMST 540 was used to provide a tau 
statistic for every possible cross-classification among the 
61 variables in the study. The tau statistic computed by 
UMST 540 was based on Siegel's (1956) formula for tau-B, and 
gave an inflated normal deviate where there were ties present 
in the rankings. This inflated normal deviate resulted from 
not making any correction for continuity prior to computing 
the normal deviate. However, the computation with UMST 540 was 
relatively simple and fast, and provided an excellent means for 
identifying those variable relationships which should be explored 
further in the last substep. 
Finally, UMST 590 was used to derive tau, S, and 
Z(s) statistics for all criterion variables cross-classified 
against each other, all hypothesis variables cross-classified 
against all criterion variables, and all other relationships 
which were indicated as possibly significant from the previous 
run based on Siegel's formula for tau-B. UMST 590 computations 
are based on Kendall's (1962) formulas for tau-B, tau-C, S, 
the variance of S, and the normal deviate of S. (See Appendix 


F for these computing formulas.) 
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PART III 


RESULTS OF THE STUDY 


Part III contains the substance of the research 
study. It is divided into three chapters. Chapter V 
presents descriptive data on the organizations in the sample 
and on the projects studied. These descriptive data consist 
of means, medians, and ranges where appropriate, and of 
distributions of responses by category where they are 
appropriate. 

Chapters VI and VII present the results of the 
statistical analysis of the sample data. The primary means 
of analysis was, as discussed in Chapter IV, the determination 
of association among variables using Kendall's rank correlation 
statistic, tau. The tables of association are included with 


the text to facilitate reader reference. 
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DESCRIPTION OF THE STUDY SAMPLE 


CHAPTER V 


Organizations in the Sample 
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The ten organizations which participated in the study 


were all located in the Twin Cities area. 


Two of the organi- 


zations were highly autonomous divisions of a large, multi- 


business corporation, 


completely unrelated. 


The ten organizations were categorized 


following industry groupings: 


1. Manufacturing. ... 


b. 


2. Non-manufacturing, 


b. 


Cc. 


d. 


e. 


Consumer products using 


continuous process 
production technology. 


Industrial products using 


assembly line/mass production 
technology ... 


Wholesale, retail trade. . 


Transportation , 
Finance, e e e 


WGP ELEY 6 whe 


Commodity merchandising 


and processing . 


Measures of Organization Size 


° 


° e 


The remaining eight organizations were 


into the 


The organization sample represented considerable range 


in terms of relative organization size. 


This fact is reflected 





265) 


in Table 1, which shows the mean, median, and range for four 


measures of organization size. 


TABLE 1 


l 
Measures of Organization Size 





Mean Median Range 
Assets (Var. 1) thousands $593,286 Sh7O,000™ S$ 60 ,000- 
(7 organizations only) $2,875,000 
Sales (Var. 2) thousands $647,000 $491,000 $ 76,000- 
(8 organizations only) $2,600,000 
Employees (Var. 3) 9,931 6,100 560- 
47,900 
Customers (Var. 4) 147 ,693 27,500 500- 
(8 organizations only) 450,000 


lmroughout Part III, the variable number is presented whenever 
a variable is discussed or entered in a table. This has been 
done to facilitate reader reference to Appendix E where all of 
the variables in the study are fully described. No particular 
significance should be attached to the variable numbers them- 
selves, however. The numbers were assigned by the researcher 
in essentially the order the variables appeared in the study 
questionnaire (Appendix B). 


Other Relevant Organization Attributes 

Several other attributes of the organization sample 
besides organization size are relevant to the present study. 
Included in this category are the reported degree of organiza- 
tional change over the last three years; the reported degree of 


centralization of the organization's MIS activities; the 
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organizational location of the manager of the information 
systems function; the organization's hardware investment and 
costs; and the size of the systems and programming staff. The 


data for all of these attributes are shown in Tables 2-4, 


TABLE 2 


Organization Change, Centralization, and 
Level of the Information Systems Manager 


Number of 
Organizational Attributes Organizations 


¢ Organization Change over Last Three Years (Var.5): 


Litstle ormmeychange . .. +. + +» © © «© 6 5 
Mittor cWahg@s . 2. 1. tw we th tt tl th tlt ls 1 
Considerable change . ..... «+ « © « « 4 


¢Centralization of Organization MIS Activities 
(Var. 8): 
All design, analysis, and programming 
performed by corporate MIS staff 
regardless of origin of project. ... 8 


Exactly as above, except that small opera- 
tions research group developed some 
ef its own projects. .... as .« 2 


« Level of Information Systems Manager (Var. 9): 
Number of hierarchical levels between 
manager directly responsible for 
information systems function and 
the top operating executive: 


OF OS | a are er 3 
eee ss. ee Ss oe 6 ee b) 
Pia |. a a 2 
* Immediate Superior of Manager of Information 
Systems: 
Top operating executive . ......ee«e 3 
Administrative vice-president ....... 3 


(or similar position) 
@Gonmerolrer. . . « « «© «6 © © * 
Market research director. ..... 
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TABLE 3 


Hardware Investment and Costs 








Mean Median Range 
Hardware Investment/Sales (Var. 6) . 94% . 90% . 10%-2.1% 
Hardware Expense/Budget (Var. 7) 35% 35% 13%-50% 
TABLE 4 


Size of Systems and Programming Staff 


Mean Median Range 


Number in Systems and Programming 50 35 7-156 
(Var. 10) 
Systems Staff/Total Employees ok. 8 oe, a267-8 27, 


(Var. 61) 
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Projects in the Sample 


This section of Chapter V deals with the descriptive 
findings relative to the twenty projects themselves. The 
variables to be discussed include those pertaining to project 
scope and size, project personnel, project procedural features, 
and project success. 

Before taking up the specific variables for which 
data were collected, several general points concerning the 
project sample are in order. First, it is reiterated that only 
projects which met the criterion of being MIS projects were 
included in the sample. Projects which were merely data 
processing applications with no management information spinoffs 
were rejected. In some instances the satisfaction of this 
criterion severely narrowed the range of projects available for 
study. In fact, in some organizations it was only after consider- 
able probing that valid MIS projects could be identified. This 
situation led to accepting projects for study which did not fall 
within the desired time and size parameters given in Chapter IV. 
Specifically, the following deviations were made from the 
criteria for project selection: 

1. It was desired that all projects in the sample 
had had at least two people who worked primarily 
on the project during its active development 
periods. Eighteen of the projects met this 


requirement, but two did not. Two projects 
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were developed entirely by one person. 
However, they were of such scope that they 
were considered to be valid components of 
the sample. 

It was desired that all projects had been 
completed between six and twenty-four 
months prior to the time of the study. 
This proved to be the most difficult 
requirement to satisfy, in that only 
fourteen projects were completed between 
November, 1968, and July, 1970. Three 
projects were completed prior to November, 
1968, the earliest of which was implemented 
in September, 1967. This means that a little 
more than three years had elapsed between 
the time the project was completed and the 
time the data were collected. While the 
period of recollection was thus over a 
year longer than the maximum recollection 
period desired, it was impossible to 
estimate how much the data were biased by 
increased forgetting of the specifics of 
the project. This same observation holds 
for all of the projects beyond the twenty- 


four month limit, and, indeed, for all of 
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the projects in the sample, excepting the 
very recent ones. 

Three projects were completed after July, 
1970, the most recent of which was imple- 
mented in October, 1970. The primary consid- 
eration with such recent projects was, of 
course, whether or not the users had had 
adequate time to evaluate the results of 

the projects. The researcher was satisfied 
that two of the projects completed since July, 
1970, did have an adequate experience base 
for fair evaluation. One of these projects, 
completed in September, resulted in a daily 
output which had provided ample opportunity 
for shakedown and appraisal. The other 
project, completed in August, had been 
evolving over a period of twelve months, 

with very close interplay between the users 
and the systems staff. In effect, this 
project was not considered completed until 
the users had indicated satisfaction with 

the products after several months of evaluat- 
ing alternative outputs. The third, and most 
recent project, completed in October, 1970, 


was studied in early December. To check on 
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the users' perceptions of the project after 
two additional months experience, a follow-up 
was made with the users in early February, 
1971. Although still two months short of 

the desired six months minimum, there is no 
reason to suspect bias enough to warrant not 


including the project in the sample. 


General Types of Projects 

As previously stated, all of the projects studied 
were management information system projects; that is, outputs 
were generated for a manager at some level which were inputs 
to his decision-making process. This general classification 
between MIS projects and data processing projects was further 
broken down into the subcategory of best fit. The subcategories 


and numbers of projects in each are shown in Table 5. 


Origin of Projects and Nature of Objectives 


Of the twenty projects studied, 65% were initiated 
by user managements. This would seem to indicate a desire on 
the part of functional managers to utilize the computer as an 
aid in carrying out their decision-making responsibilities. 
However, the reported objectives in initiating projects did 
not always seem to be as clear and specific as they might 
have been. These observations are based on the distributions 


of project origins and project objectives shown in Table 6. 
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TABLE 5 


Types of MIS Projects Studied 





Number of 
Projects 


Types of MIS Projects of Type 


— 


NN 


WW 


Models: projects involving management 

science techniques such as simulation, 

mathematical programming, or forecasting; 

generally projects aimed at providing 

inputs to planning processes .....e.e.- 4 


Data processing spinoffs: projects 
which evolved from operational control 


systems at the lower levels of the 

organization; generally spinoffs from 

accounting and logistics systems, and 

aimed at providing input to control 

processes through monitoring, triggered, 

or demand reports (see Blumenthal, 1969, 

Lo a a nr ) 


Data collection and analysis: projects 
which were developed from the ground up 


with specific uses and requirements; 

generally involved setting up means to 

collect desired information, creating 

the necessary data base, and developing 

analytical routines to manipulate the 

data base; these projects aimed at 

providing inputs to both planning and 

control processes; an example of such a 

project would be a marketing intelligence 

Syoccums «os s Sf ots 6 6 6 6 6 6 8 © © 6s «6 7 
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TABLE 6 


Project Origins and Objectives 





Number of 


Projects 


Origin of Project (Var. 11) 


User. ® e ® e ® ® ® e @ ® e e e e e e e e ® e 13 
Top level management. .... +e « « © « « © e 3 
Information systems staff... 2. 6 2 2 © 6 « 4 


Project Objectives (Var. 12) 
Specific, measurable, written objectives. .. 
Specific, non-measurable, written objectives. 
General, clear, written objectives, ..... 
General, unwritten objectives ......e... 
Rather vague objectives . . . . «6 « « «© « « 


WHA 


Project Scope and Size 


The following variables relate to the size and scope 

of the projects in the sample: 

Complexity - (Var. 13) 

There was no objective measure of comparative 

complexity of projects in the sample. The 

ratings of project complexity were entirely 

relative to the individual meen leaders and 

the systems environments in their organizations. 

For this reason, the distribution given below is 

not the distribution of complexity of all sample 

projects based on some objective scale applied 


to all projects; rather, it is a distribution of 
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the ratings of complexity assigned by each 
project leader relative to his own experience: 
Highly complex . . 2. 2. 2. « + of 
Above average complexity .. .5 
Average complexity ..... .8 
Below average complexity .. .2 
Very low complexity. .... 2 
Size and composition of project staffs 
Table 7 provides a breakdown of the composition and 


size of project staffs for the projects in the sample: 


TABLE 7 


Composition of Project Staffs 


Mean Median Range 


Number on Project (Var. 17) j 5.0 4 1-12 
Number of Analysts (Var. 18) ies 1 1-3 
Number of Programmers (Var. 19) Pap. 2 1-8 
Number of Users = N=l3 (Var. 20) 1.8 Z 1-3 


Outside consultants used on three projects 
Time spent on projects 
Table 8 contains data on the elapsed time and the man 


months spent on the projects in the sample: 
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TABLE 8 


Time Spent on Projects 





Mean Median Range 
Elapsed Months (Var. 21) 127 10.5 2-48 


Man Months (Var. 22) 22.0 10.0 4-87 





Attributes of Project Personnel 


The following variables relate to the education, 
systems experience, and organization experience of those who 
worked directly on the projects in the sample: 

Systems experience 
Table 9 contains data on four measures of the 
systems experience of those who worked directly 


on the projectsstudied: 


TABLE 9 


Systems Experience of Project Personnel 





Mean Median Range 


Systems Experience of Project Leader-- 
months (Var. 24) 66.2 43.5 0-300 


Mean Systems Experience of Project 
Personnel Including Users-- 
months (Var. 25) 47.3 28.5 8-180 


Mean Systems Experience of Project 
Personnel from MIS Staff Only-- 
months (Var. 26) 5265 375 11-180 


Two or More Years Systems Experience-- 
proportion (Var. 27) 587. 50% 0-1007% 
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It should be noted that the values given in Table 9 
for Variables 25, 26, and 27 are not those for all 
project personnel in the sample. Rather, they 
are the values derived from the means or proportions 
that Variables 25, 26, and 27 represented for each 
project. Thus, 47.3 months is the mean of twenty 
project means, not the mean systems experience for 
all individuals who worked on projects in the sample. 
Impact of systems experience - (Var. 28) 
With respect to the impact of systems experience on 
project success, project leaders generally indicated 
that such experience was a contribution to project 
success, and lack of systems experience hindered 
success somewhat. However, as the following 
distribution of Variable 28 shows, 25% of the 
project leaders felt that prior systems experience 
was of no importance one way or the other to 
project success: 

Experience critical to success .... .4 


Experience contributed somewhat 
GQnOUCCeSSe es. 2 . cee. . eons 


Experience of no importance. ..... .939 


Lack of experience of some staff 
members detrimental . ...... .3 


Education 
Table 10 contains data on three measures of the 


educational levels of those who worked directly 
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on the projects studied: 


TABLE 10 


Formal Education of Project Personnel 





Mean Median Range 


College Degree - 
proportion (Var. 30) 6 5% 75% 0-100% 


Two Years College - 
proportion (Var. 31) 797 87%  33-100% 


Mean Years Formal Education 
(Var. 32) P53 L533 13-18 


As with systems experience, it should be noted that 
the above values are derived from project statistics, 
not individual measures. 
Organization experience 
Table 1l contains data on two measures of the 
experience project personnel had in their respecte- 
ive organizations at the beginning of the projects 
studied, 

TABLE 11 


Organization Experience of Project Personnel 





Mean Median Range 


Mean Years Organization 
Experience (Var.33) 6.75 6.75 1.5+20.0 


Two or More Years Organization 
Experience - proportion (Var.34) 69% 69% 25-100% 
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As with systems experience and education, the above 
values are derived from project statistics, not 


individual measures. 


Turnover - (Var. 29) 

Of the twenty projects in the sample, only three 
experienced any turnover among the project staff. Two of 
these projects had a turnover rate of 50%, while the third 
project's turnover rate was 67%. For the three projects 
which experienced staff turnover, the project leader's 
appraisal of the impact of this turnover on project success 
was distributed as follows: 

Very detrimental ...... .l 

Somewhat detrimental .... .1l 


Contributed to success ... .l 


Procedural and Organizational Attributes of Projects 


Tables 12 and 13 contain breakdowns of the study 
sample according to several organizational and procedural 
attributes of the projects. In addition to the information in 
these tables, several amplifying or explanatory comments 
concerning the attributes shown are in order. 

As can be seen from Table 12, thirteen of the projects 
studied (65%) were developed by project teams comprised of user 
representatives working with information systems staffs. Of 


those thirteen, two project teams had user representatives 
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As with systems experience and education, the above 
values are derived from project statistics, not 


individual measures. 


Turnover = (Var. 29) 

Of the twenty projects in the sample, only three 
experienced any turnover among the project staff. Two of 
these projects had a turnover rate of 50%, while the third 
project's turnover rate was 67%. For the three projects 
which experienced staff turnover, the project leader's 
appraisal of the impact of this turnover on project success 
was distributed as follows: 

Very detrimenctaY os. . ee lk 

Somewhat detrimental .... .l 


Contributed to success ....l 


Procedural and Organizational Attributes of Projects 


Tables 12 and 13 contain breakdowns of the study 
sample according to several organizational and procedural 
attributes of the projects. In addition to the information in 
these tables, several amplifying or explanatory comments 
concerning the attributes shown are in order. 

As can be seen from Table 12, thirteen of the projects 
studied (65%) were developed by project teams comprised of user 
representatives working with information systems staffs. Of 


those thirteen, two project teams had user representatives 
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assigned full time during the analysis and design phases; the 
remaining eleven project teams had user representatives working 
on the projects on a part-time basis. Further, for eleven of 

the thirteen projects where a team was utilized, the user 
representatives who participated on the teams were also respon- 
sible for implementing the completed projects in their respective 
departments, and for maintaining liaison with the information 
systems staffs after implementation. For the other two 

projects, team participants during development were not the 

ones responsible for implementation and continuing liaison. 

For the thirteen projects which were developed by 
teams, the mean value for the proportion of total project man- 
months accounted for by user personnel (Var. 15) was 18%; the 
median value was 18.5%; and the range was 4-51%. 

Finally, project leaders generally felt that user 
representatives on project teams made valuable contributions 
to project success. Nine of the project leaders who headed 
teams appraised user member contributions as very great, as 
critical to project success. The other four project leaders 
felt that user members made some contributions, and that the 
projects would not have turned out as well as they did without 
the users. 

With respect to project documentation, formal standards 
for documentation were considered to exist even if such 


standards were applicable to only the programming phase of a 
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project. It should also be pointed out that the quality of 
project documentation was not measured against any universal 
objective standard or scale. Rather, the documentation quality 
distribution shown in Table 12 reflects the appraisals of 
individual project leaders as to how good they thought the 
project documentation was for their respective projects. 

The degree of user participation in the analysis 
and design of each project was appraised by both user personnel 
and the project leader. The amount of agreement between those 
appraisals will be taken up in a subsequent chapter. However, 
it is worth noting at this point that there was a tendency 
for fairly high levels of user participation in the projects 
studied, whether viewed from the users' side or the project 
leader's perspective. This observation is supported by the 


statistics in Table 13. 


TABLE 12 


Project Procedural and Organizational Attributes 





Number of 


Projects 


Project Team (Var. 14): 
Yes e e e e e e e e e e & e e e e e e e e e e iS 


No e e e e e e e e e e e e e e e e e 6 e e e 7 


Combination Analyst/Programmer (Var. 23): 
Comb ina tion e @ e e e e e ® e e e e e e e e e 9 
Separa te e e e e e t e e e e e e e e e e e e e ll 


Documentation Standards (Var. 35): 
Yes e a e e e e e e td] e e e @ e e e e e cc) e e 16 


No. e e ° e e e e e e e e e e e e e e e e e e 4 
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TABLE 12 (Cont.) 


Project Procedural and Organizational Attributes 





Number of 


Projects 


Quality of Documentation (Var. 36): 
Pip meuet ty. «se os + oe + pills © ce 
Recqua te, Soe: Ga. ee a.m. 
Somewhat inadequate .......+-+«-ec.see. 
Mowequawity . . 5 6 ee se ee tw tlt lt lw 8 


rm in © OV 


Project Control (Var. 42): 
Formal or informal report of progress 
against plan required of project 
staff at least monthly. . ...... +. « « 11 


No regular progress reporting 
required of project staff. - « « « «© « «© « « » 9 


Programming Language (Var. 43): 
ne, ee A ee i 
PON. F 5 e.g ee ee, ae, es 2 
Co ee rae 3 


Generalized Software Problems (Var. 44): 
No problems ...... ° 
Pn PEOD OMG cs se Sine, eee sue wyene « 4 
Major problems. ... . ° 





TABLE 13 


User Participation 


Mean Median Range 


User Participation-PL (Var. 41) 
possible range: 3-15 LL. 12.5 4-15 
User Participation-User (Var. 46) 


possible range: 40-200 153 162 83-200 
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Project Success 


It was stated in Chapter III that four criteria of 
project success were defined for the study: time, cost, user 
satisfaction, and computer operations problems, The performance 
of the projects on each of these criteria is shown in Table 14. 
In addition to what is shown in the table, the following comments 
concerning the criteria are relevant: 

Time 

Of the twenty projects, six were completed within 
the times estimated; the remaining fourteen 
exceeded their time estimates by varying amounts 
up to 900%. Of the six projects which were 
completed on time, overtime was required to do so 
for three of them. One of the three on-time 
projects which had considerable overtime also 

had substantial programmer resources added to 

the staff to meet the time deadline. 

Cost 

An interesting finding regarding project cost 

was that nine of the twenty projects in the 
sample had no cost budget. No cost estimates 
were ever made, or at least ever recorded or 
referred to later, for nearly half of the 
projects studied, Since it was impossible to 


determine if a project was completed within its 
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cost budget if such a budget never existed, 

the cost data for these projects were estimated, 
based on manemonths data and estimates of 
computer test time. This seemed to be a 
reasonable approximation to project cost, 

since by far the largest part of project cost 
was accounted for by manpower cost, which, 

in turn, was a direct function of man-months 
spent on a project. Where program test time 
was determined, through discussions with the 
project leader, to be of any consequence in 

the total project cost, a factor for computer 
cost was included in the cost estimates. 

Using, then, the best estimates that could be 
devised for project cost data, it was found 
that only two of twenty projects were completed 
within their cost budgets. The remaining 
eighteen projects exceeded their cost estimates 
by varying amounts up to 500%. 

User satisfaction 

Since five separate factors were aggregated 
together to form the measure of user satisfac- 
tion (Var. 54), the range of possible values 
for this measure was 50-250, with 50 represent- 


ing extremely low user satisfaction and 250 
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representing extremely high user satisfaction. 
As the statistics in Table 14 indicate, a 
fairly high level of user satisfaction existed 
for most projects in the sample. If the sum 
of the middle of the five possible ratings on 
the scale for each of the five factors was 
taken as an ‘average’ level of user satisfaction, 
the resulting ''average'' value would be 150. 

It can be seen that both measures of central 
tendency shown in Table 14 for the actual 
sample well exceeded that hypothetical value, 
Computer operations problems 

In general, the projects in the sample were 
successful in terms of not creating problems 
in the computer operations function. With a 
five point scale, ranging from very serious 
problems at the low end to no problems at the 
high end, a hypothetical average would have 
fallen in the middle of the scale, or at a 
value of 3, which represents moderate problems 
for computer operations. However, the mean 
value for computer operations problems (Var. 59) 
in the sample was 3.95, and there was no 


project in the sample rated below 3 on the scale. 
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The distribution of the ratings on this 
criterion was as follows: 

No problems. ...... 9 

Very minor problems. .. 9 


Moderate problems. .. . 6 


TABLE 14 


Project Success 


Mean Median Range 
Actual Time/Estimated Time (Var. 48) 209.6% 139.5% 75+900% 
Actual Cost/Estimated Cost (Var. 51) 194.7% 151.5% 82-500% 


User Satisfaction-User (Var. 54) 175.8 192.5 80-220 
(possible range: 50-250) 


Computer Operations Problems (Var. 59) 3.95 4.0 325 
(possible range: 1-5) 


summary 

In summary, it can be seen from a review of the 
descriptive information presented in this chapter that the 
study sample represented considerable diversity among organiza- 
tions and projects. The organizations in the sample were from 
varied industries; they represented a wide range in terms of 
size; and they differed in varying degrees on several other 
organizational attributes relevant to the study. With respect 


to the individual projects studied, there were three general 
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types of MIS projects: models; projects that were spinoffs 
from data processing applications; and projects which were 
developed from the ground up to provide data collection and 
analysis capability in specified planning and control areas, 
The twenty projects represented substantial diversity in 
terms of size and scope; procedural and organizational 
attributes; and, finally, performance on the criteria of 


success, 


sours 


CHAPTER VI 


TESTS OF HYPOTHESES 


Introduction 
This chapter and the one which follows are devoted 
to presenting the results of the analysis of the sample data. 
As indicated in Chapter IV, the primary form of statistical 
analysis was the computation of Kendall's rank correlation 
statistic, tau. This measure of the degree of relationship 
between the rankings of two variables was tested for statistical 
significance in all cases. 
The number of possible cross-classifications, and 
resultant relationship statistics, with 61 variables was 3660. 
To avoid burdening the reader with all of these statistics, 
and to conserve computation time and cost, the steps discussed 
in Chapter IV were followed, The results of the data analysis 
have been further limited here by presenting only the following: 
1. Relationships among criterion variables; 
all tau statistics presented regardless 
of significance levels. 
2. Relationships of hypothesis variables 
to each of the four criteria of success; 
all tau statistics presented regardless 
of significance levels. 
3. Relationships of criterion variables to 


other non-hypothesis variables which were 
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significant at the .10 level or beyond. 
4, Relationships of hypothesis variables 
to other nonecriterion variables which 
were significant at the .10 level or 
beyond, 
5. Other relationships among variables 
felt by the researcher to be relevant 
to the study, which contributed to 
understanding of the overall conclusions 
drawn, and which were significant at the 
~10 level or beyond, This last category 
is presented in Chapter VII. 
All tests for the significance of the relationship 
between the rankings of two variables were made with the .10 
level as the cutoff. Any relationship with a probability of 
chance occurrence greater than .10 was not considered to be 
significant. Since the direction of expected relationship 
was stated in only those cases where a hypothesis was tested 
(category 2 above), all significance tests were two-tailed 
tests except those involving the relationship of a hypothesis 


variable to a criterion variable. 


Relationships Among Criterion Variables 


At the time this study was conceived, it was assumed 
that the four criteria to be used were independent. Otherwise, 


all four would not have been included in the study. However, 
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no hypothesis to this effect was set up. For this reason, 
all tests of relationship among the four criteria are two- 
tailed tests. 

It can be seen from Table 15 that there were no 
relationships among any of the criteria which reached signifi- 
cance at the .10 level. This supports the assumption of 


independence among the four criteria used in the study. 


TABLE 15 


Relationships among Criterion Variables* 


User Satis- Operational 
Cost (Var. faction (Var. Problems (Var. 


A A et i 


Time 

(Var. 48)/.257/1.534|.124 -064} .359 | .720 778 
Cost 

(Var. 51) O91) .522 | .604 916 
User Sate 

isfaction 

(Var. 54) .104 





tan values shown for tau in all tables are tau-C unless a given 


value is prefixed by B, in which case that value is tau-B, 


See Appendix F for differences in computation of tau-B and 
tau-C, 


P(Z) value shown includes both tails of the normal distribution. 
For all tables, the value shown under P(Z) represents the 
chance probability, one-tailed or two-tailed, as indicated, 
of having a normal deviate for the S statistic as large as 
the one shown under Z(s). 
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The strongest relationship between any two criterion 
variables was the one between user satisfaction-user (Var. 54) 
and operations problems (Var. 59), with a tau of .345. This 
relationship had a chance probability of .104, which almost 
reached the .10 cutoff level established. Although not signifi- 
cant based on the arbitrary .10 cutoff used in the study, there 
did appear to be a meaningful relationship between how satisfied 
the user was and how successfully the project had been implemented 
by computer operations. Where there were substantial problems 
for computer operations in running particular programs, or in 
meeting certain schedules, there also tended to be lower user 
satisfaction. It cannot be assumed that computer operations 
difficulties caused user dissatisfaction, although this did 
appear to be the case with two of the projects studied, Rather, 
the conclusion drawn was that those projects which created 
problems for the computer operations area were also somewhat 
unsatisfactory in the eyes of the users of the outputs. In 
other words, there was a tendency for shortcomings in the 
systems development work for these projects to be reflected 
in both inadequate planning for impact on computer operations 


and poor response to user information needs. 


Tests of Hypotheses 


Tables 16-19 provide a complete picture of the relation- 


ship between each hypothesis variable and each criterion variable. 





TABLE 16 


Success in Terms of Time 


Actual Time/Estimated Time (Var. 48) 


Rank 


12 


13 


14 


16 


is 


20 


23 


31 


32 


33 


34 


* - Significant at the .10 level or 


User Participation-User 
Measurable Project Objectives 
Project Team 


Level of Information Systems 
Manager 


Two or More Years Systems 
Experience 


Documentation Standards 
Project Control 
Turnover 

Originator 
Centralization 


Two or More Years Organization 
Experience 


High Level Programming Language 


Mean Years Formal Education 
Combination Analyst/Programmer 
Systems Staff/Total Employees 


Hardware Investment/Sales 


Var. 


46 


12 


14 


27 


35 


42 


29 


11 


34 


43 


32 


23 


61 


Tau 
-,181 
-,219 


-.130 


-.292* 


. 203 
-.380* 
-.153 

082 


-,037 


wag7 * 
-,315* 
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-,063 
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TABLE 17 
Success in Terms of Cost 


Actual Cost/Estimated Cost (Var. 51) 


* - Significant at the .10 level or beyond = one-tailed. 


Rank Var. Tau Z(s P(Z 
l User Participation-User 46 ~154 894 185 
2 Measurable Project Objectives 12 - 100 ~ 503 - 309 
3 Project Team 14 20 794 214 
7 Level of Information Systems 

Manager 9 -.240 1.107 ~ 135 
8 Two or More Years Systems 
Experience 27 -.104 ~ 596 e295 

12 Documentation Standards 35 -.200 899 ~185 

13 Project Control 42 -.220 1.109 134 

14 Turnover zo 030 ~ 158 437 

16 Originator Ll 187 e930 7 

17 Centralization 8 Not tested by this sample 

20 Two or More Years Organization 

Experience 34 ~055 2295 - 384 

23. High Level Programming Language 43 -.097 omg 302 

31 Mean Years Formal Education 32 - 108 567 Zen 

32 Combination Analyst/Programmer 23 -.010 000 - 500 

33 Systems Staff/Total Employees 61 .263* 1.491 » 069 

34 Hardware Investment/Sales 6 -.023 .099 461 
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TABLE 18 
Success in Terms of User Satisfaction 


User Satisfaction-User (Var. 54) 


Rank Var. Tau Z(s P(Z 
1 User Participation-User 46 o440* 2.618 005 
2 Measurable Project Objectives 12 -.081 403 345 
3 Project Team 14 - 080 278 - 390 
7 Level of Information Systems 

Manager 9 105 464 eee 
8 Two or More Years Systems 
Experience 27 -.126 0/29 0 234 

12 Documentation Standards 35 2090 0379 Pe 5 8 

13. Project Control 42 -.207 1,040 ~ 149 

14 Turnover 29 -.217* 1,480 069 

16 Originator 11 0420* 2,132 017 

17 Centralization 8 Not tested by this sample 

20 Two or More Years Organization 

Experience 34 0293* 1,477 069 

23 High Level Programming Language 43 067 346 - 366 

31 Mean Years Formal Education 4 e222 1.201 115 

32 Combination Analyst/Programmer 23 ~420* 1.561 » 060 

33 Systems Staff/Total Employees 61 o229*% 1,293 098 

34 Hardware Investment/Sales 6 057 «296 ~ 383 


* = Significant at the .10 level or beyond = one-tailed. 
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TABLE 19 


Success in Terms of Computer Operations 


Computer Operations Problems (Var. 59) 


Rank Var. Tau Z(s Rd Z 
1 User Participation-User 46 180 ~820 » 206 
2 Measurable Project Objectives 12 -.007 - 000 2 200 
3 Project Team 14 -.050 170 433 
7 Level of Information Systems 

Manager 9 Be,285* 1.340 090 
8 Two or More Years Systems 
Experience 27 -.255 1.176 ~120 

12 Documentation Standards 35 -.170 811 210 

13. Project Control 42 -.060 261 397 

14 Turnover 29 +B .000 - 000 - 500 

16 Originator ll B=-.147 -665 2 254 

17 Centralization 8 Not tested by this sample 


20 Two or More Years Organization 


Experience 34 157 ~/07 241 
23. High Level Programming Language 43 Be.157 2695 ~ 243 
31 Mean Years Formal Education 32 ~330* 1,542 062 
32 Combination Analyst/Programmer 23 060 . 204 420 
33 Systems Staff/Total Employees 61 037 143 443 
34 Hardware Investment/Sales 6 e2oe 1.063 144 


* - Significant at the .10 level or beyond - one-tailed. 
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Each of these tables shows the relationships between all sixteen 
of the variables representing hypotheses to be tested and one 
criterion variable. The sixteen hypothesis variables are shown 
in the same order as in the ranking in Chapter IV; the number 
to the left of each variable is the rank of that variable from 
the original list (Figure 2, Chapter IV). 

Rather than merely repeat in narrative form what is 
contained in Tables 16-19, the comments here will be restricted 
to general observations about the findings shown in the tables, 
and discussions of those hypotheses which were significantly 
related to at least one criterion: 

1. Each of the hypotheses was represented by one 
variable in the statistical analysis. Appendix 
E should be consulted for a description of what 
each variable is a measure of, and how it is 
scored. Of the sixteen hypotheses, all but 
one were tested by the data from the sample. 
Variable 8, representing the degree of centrali- 
zation of the information systems function, was 
dropped from the analysis because of the lack of 
spread on the variable among the organizations in 
the sample. Eight organizations were scored at 
the highest level of centralization, and the 
remaining two were at the next to highest level, 
a result solely of small operations research 


groups which did some programming work. 





2. 


«00 


Of the fifteen hypotheses which were tested, 
five were found not significantly related to 
any criterion. Those five were: 


2 - Measurable project objectives - Var. 12 


3 - Project team Var. 14 
8 = Two or more years systems 

experience Var. 27 
13 - Project control Var. 42 
34 - Hardware investment/sales Var. 6 


The fact that those hypotheses which were not 
significantly related to any criterion were from 
the top, middle, and bottom of the ranked listing 
is of of some interest. At least for the sample 
involved in this study, those factors believed 

by the SMIS conference respondents to be most 
crucial to MIS project success did not prove to 

be so critical in the case of two of the three 

top ranked factors. Also of considerable interest 
was the lack of significant relationship between 
reported project control efforts and any criterion 
of success used in the study. This last finding 
will be explored further at a later point. 

Of the ten hypothesized relationships which were 
found to be significant at the .10 level or 
beyond, four were cases where a hypothesis variable 


was significantly related to only one criterion 
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variable. The remaining six hypothesis 
variables were related to two or three 
criterion variables, 

4, Looking at the tables from a criterion 
perspective, the time success criterion 
was Significantly related to more hypothesis 
variables (7) than was any other criterion 
variable. User satisfaction was signifi- 
cantly related to six hypothesis variables, 
computer operations success to two, while 
cost success was significantly related to 


only one hypothesis variable, 


Hypotheses Significantly Related to Project Success 


Each of the hypothesis variables which was significantly 
related to at least one criterion of project success is dealt 
with briefly below: 

l - User Participation-User (Var. 46). As hypothesized, 
the higher the level of perceived user participation 
in design and development of a project, the greater 
the success of that project as viewed by the user. 
The statistical relationship between these two 
variables was very strong in the sample. However, 
perceived user participation in project develop- 
ment was not significantly related to any of the 


other three criteria of success, 
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7 - Level of Information Systems Manager (Var. 9). 


The higher the level of managers of the infor- 
mation systems departments in the organizations 
sampled, the less successful projects were in 
terms of time, and the more successful they were 
in terms of computer operations. Although there 
were no data from the study which explained these 
relationships very satisfactorily, it could be 
that the higher the level of the top computer 
executive in the organization, the less the 

felt pressure in the information systems function 
to meet time budgets. With respect to computer 
operations problems being less where the top 
computer executive was at a higher level in the 
organization, one possible explanation would be 
the apparent greater attention to computer 
operations requirements when the information 
systems function is accorded higher status in 
the organization. Put differently, where the 
organization has placed emphasis on using the 
computer effectively, as evidenced by locating 
the top computer executive close to top manage- 
ment, there may be more emphasis on assuring 
that computer operations requirements are 


considered in systems design. 
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12 - Documentation Standards (Var. 35). Where 


14 


formal documentation standards were reported 

to have been applicable to projects in the 
study, the time success tended to be greater 
than where there were no such standards. It 

was impossible to determine whether the documen- 
tation standards themselves actually contributed 
to better performance in terms of time, or the 
existence of such standards merely reflected 

an atmosphere of greater control over informa- 
tion systems activities. In any case, this 
finding should be interpreted with caution due 
to the great extent of ties in the ranking of 
the dichotomous documentation standards variable. 
Turnover (Var. 29). As was pointed out in 
Chapter V, only three of the projects in the 
sample experienced any turnover of project 
personnel. Therefore, the negative relationship 
of turnover to user satisfaction must be inter- 
preted very cautiously due to the great extent of 
ties in the turnover ranking. A close review 

of all the variables for those three projects 
with turnover led to the conclusion that the 
negative relationship with user satisfaction 


was a chance one, and should not be given much 
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credence. The reason for this conclusion 

was that in the case of only one project 

did low user satisfaction follow at all 

from project staff turnover. In that one 

case, the turnover was among key user repre- 
sentatives on the project team. 

Originator (Var. 11). The hypothesis that 
success is greater for projects originated 

by users, as opposed to those initiated by 

top management or the information systems 

staff, was confirmed where success was viewed 

in terms of user satisfaction. This would 
appear to stem in part from a greater feeling 
of participation among the users who originated 
projects, and, in part, from a greater clarity 
in what the users wanted when they initiated 
projects themselves. Where the users knew 

what they wanted, and participated in develop- 
ing it, their satisfaction with the results 
tended to be greater. 

Two or More Years Organization Experience (Var. 34). 
The organization experience of the project staff 
was significantly related to two criteria: time 
and user satisfaction. In the case of time, the 


greater the organization experience of the project 
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staff, the poorer the time performance. 

This would seem to reflect a greater tendency 
for those who knew their organizations well 

to spend more time delving into the various 
ramifications and possibilities with projects 
once development was underway. This would 
likely draw out the time spent on the projects 
beyond what was originally anticipated. In 

the case of user satisfaction, the positive 
relationship to organization experience of 

the project staff indicated that the greater 
the proportion of those working on a project 
who understood the organization they were serving, 
the better the results in the eyes of the users. 
It is worth noting here that organization 
experience was not related at all to the 
measure of user participation. It might 

have been assumed that high organization 
experience would have reflected high propor- 
tions of user representatives on project teams, 
which, in turn, would have contributed to 

high levels of perceived user participation. 
Since user participation was highly related 

to user satisfaction, the assumption would 


then be that this last relationship was 
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really what was being picked up in a 
relationship between user satisfaction 

and organization experience of the 

project staff. This was not the case, 
however. 

High Level Programming Language (Var. 43). 
From the data in the sample, the hypothesis 
of greater project success with higher level 
programming languages was confirmed with 
respect to the time criterion. Where COBOL 
was the programming language used, the 
projects were completed in less time 
relative to estimates than where FORTRAN or 
assembly language was used. However, it 
should be noted that the types of projects 
were different where FORTRAN was used, and 
this in itself influenced the time performance. 
FORTRAN was used with modeling applications 
which were evolutionary in nature. Through 
trial and error processes, the models were 
developed to the point the users were satis= 
fied with the results. This situation 
resulted in very extended development periods 
for these projects. 

Mean Years Formal Education (Var. 32). The 


mean education level of the project staff was 
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very strongly related to the time criterion 
of success. What this indicated was that 

the higher the education of those who 

worked on projects, the poorer the perform- 
ance in terms of time. The only apparent 
explanation for this relationship was a 
tendency for those with more formal 

education to delve more deeply into 

possible enhancements and embellishments 

on the projects they worked on. Again, the 
influence of modeling applications should 

be considered in this connection. Those 
engaged in modeling applications generally 
had high levels of formal education. It 
will be recalled that the modeling projects, 
because of their evolutionary nature, were 
drawn out and usually way over the original 
estimates for time to completion. 

Also of significance was the relationship of 
project staff education levels to computer 
operations success. One explanation for 

this relationship appeared to be the tendency 
for those with the greatest amounts of formal 
education to make provisions for the efficient 


computer implementation of their projects. 
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32 - Combination Analyst/Programmer (Var. 23). 


33 


Where combination analyst/programmers were 
used to develop projects, the time performance 
was relatively poor, but user satisfaction was 
relatively high. While no explanation was 
readily apparent for the poor time performance 
where combination analyst/programmers were 
used, the high user satisfaction seemed to 

be attributable to the ability of the user 

to look to one person for any problem that 
arose. Where the user had to deal with an 
analyst in some cases, and with a programmer 
in others, there appeared to be user frustra- 
tion and less satisfaction with the project 

as a whole. 

Systems Staff/Total Employees (Var. 61). 
Although they were all rather weak relationships, 
the relative size of the organization's systems 
staff was related to three separate criteria. 
Positive relationships to both the time and 
the cost criteria would seem to indicate that 
there was more slack in those organizations 
with large systems staffs. Where large staffs 
existed, the pressure to meet time and cost 


budgets or estimates did not seem to be as great. 
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As a consequence, time and cost performance 
were poor. However, user satisfaction was 
higher for those projects developed in organi- 
zations with larger systems staffs. Taken 

all together, the pattern would seem to be one 
of substantial organizational commitments to 
effective use of the computer as an aid to 
managerial decision-making, manifested through 
relatively large systems staffs which were 
oriented toward giving the users what they 


wanted at the expense of time and cost overruns. 


Further Comments Concerning the Criteria of Success 


Data were collected for several other variables 
besides the criteria and the hypotheses to be tested. To provide 
a more complete picture, all significant relationships between 
these other variables and the criterion variables are shown in 
Tables 20-23, These tables, and certain relevant non-quantitative 
findings, are discussed in this section. 

Time Success (Var. 48). Table 20 contains those 
relationships between time success and other variables in the 
study which were significant at or beyond the .10 level. As 
might be expected, the projects with the greatest elapsed time 
had the poorest time success. Also, the higher the proportion 
of college degrees on project staffs, the poorer the time perform- 
ance. This is merely another way of looking at the educational 


level, which was discussed earlier. 





ee Od RS es ND be -— <—- 
ate eT ee -— ee ewe 
—— ie ie ee ee ee 
o— + ee eee oe PO 
ientnestnn"enene anil bs i in ale 
tn DD Tbh —_o eames 
ee ee 
a a @ = ewe oo cot le 
~*~ at Ok ee we eet me mm fe, 
|] aE, ee eS eee & «Oe 
—_——— —P wee ee ee ee 
———— a A mon we cme wake ‘ae 
—imeitci wd = Gu jaee of 
ne i —é,~) —_ ai aie’ 

















-109- 


TABLE 20 
Actual Time/Estimated Time (Var. 48) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
College Degree 30 2950 So 2 or ~0014 
Elapsed Months 21 Re 3.081 002 
User Management Interest 39 -.36/7 1.864 062 
Time Success =PL 49 -.437 22028 020 





From the data in the study, it appeared that high 
interest and involvement of top level user management contributed 
to better time performance, Where the top management levels of 
user organizations were actively engaged in promoting projects, 
those projects seemed to get completed faster relative to time 
estimates. This finding has interesting implications which 
tie in to some comments to be made later concerning the reward 
structures for systems people in organizations. 

Another finding shown in Table 20 was the accuracy 
of project leader's perceptions concerning the success of their 
projects in terms of time. It should be borne in mind, however, 
that this relationship is based on asking the project leader 
only how successful he felt the project was in terms of time, 
with no consideration given to his views of other aspects of 


project success. This point, too, will be dealt with later 
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at greater length. 

Cost Success (Var. 51). Table 21, consisting of 
only one entry, provides the same sort of confirmation of 
the project leaders’ perceptions about project cost success 
as described just above for time success. Again, this point 
will be dealt with later in the context of global project 
success, and what that seemed to be comprised of in the 


eyes of project leaders. 


TABLE 21 
Actual Cost/Estimated Cost (Var. 51) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 


Cost Success=PL 52 -.644 3.444 . 0006 


In addition to the questions asked of project leaders 
which were tied to some kind of response scale, there were two- 
open-ended questions which permitted project leaders to state 
in their own words the reasons they felt their projects came 
out poorly on time and/or cost performance. These comments 
were reviewed by the researcher and classified according to 
eight response categories. The categories, and the number of 
projects falling into each, were as follows: (there were 


multiple responses for most projects) 
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1. Poor initial estimates; especially 

inadequate was the time allowed for 

implementation... wi 2 2 « «© © © © © © wo ow lO 
2. Inexperience of the project staff 

with this particular type of applica- 

tionvor lamegitiage . 3d. . . Bue 2c ew ewe / 
3. Key people on the project were doing 

several things at once; competing 

demands for their time caused delays .....4 
4, The project was allowed to evolve over 

a long period of user learning and 

growth; no attempt was made to freeze 

requirements at one point and then 

consider the job done when those 

requirements were programmed . .....e«-.«-e«e 4 
5. File handling problems; delays caused 

by waiting for data base development 

OF GeceosmOlLitya. « Ge 0 GMC « 6.006) ensues S 
6. Poor computer test turn-around time. ..... 2 


7. Turnover Of project Sta@er. . . 6 «© «© ew we wo ow 


8. Project was too large and complex 
to be managed e e e e e e e e e e @ e e e e e ° 2 


From the above, it would appear that the greatest 
problem in meeting time and cost budgets, at least, in the 
eyes of project leaders, was poor estimates of time and cost 
in the first place. Actually, most of the other categories 
are related in some way to the first one. In estimating time 
to complete a project, the experience of the project staff 
should have been considered, as should the expected availability 
of computer test time, and so forth. This is not to say that 
every contingency could have been foreseen and provided for 


in making initial estimates, but it would seem that some factors 
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which did cause delays were not given adequate consideration 
at the time estimates were made. 

One final point concerning project leaders’ comments 
about time and cost performance is worth mention. In two 
cases, the project leader had no direct authority over programmers. 
This created communications problems which resulted in considerable 
confusion as to what was to be programmed, with attendant delays. 
Both of the projects were in the lower half of every criterion 
ranking. 

User Satisfaction (Var. 54). A close scrutiny of the 
users' ratings of their satisfaction with the results of the 
projects in the study revealed several interesting points. It 
will be recalled that the measure of user satisfaction was a 
composite variable, made up of five separate items which were 
highly intercorrelated. However, one of the components of 
the user satisfaction variable, a measure of implementation 
problems, was also treated as a separate variable (53) for 
comparative purposes. As Table 22 shows, the users viewed 
the ease or difficulty of implementing a completed project 
as a very important aspect of the perception of project success, | 

igi there is bias in the value of tau shown in Table 
22, due to the inclusion of the implementation problems 
variable in the user satisfaction variable, it is apparent 


that implementation was a large factor in the minds of users 
in evaluating the success of projects. 
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TABLE 22 
User Satisfaction-User (Var. 54) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Implementation Problems- 

User 53 .720 ee Ne id 0004 
User Satisfaction-PL 37 656 3.584 2 0004 
Project Success-PL 56 607 3.124 .002 
Computer Operations 

Cost 60 pte? e796 ~074 
Satisfaction of Objectives- 

PL 35 320 Pooo? 092 
Man Months jist -.280 1,668 096 





It is also noteworthy that the variable representing 
the project leader's perception of implementation problems (58) 
was not significantly related to user satisfaction. This last 
observation will be pursued further at a later point, but it 
is apparent that users viewed implementation problems differently 
than project leaders viewed them. This conclusion is supported 
by the very strong relationships between user satisfaction and 
three variables ee the project leader's evaluation 
of project success. These three variables, shown in Table 22, 
were the project leader's perception of how satisfied the 
users were with the project results (57), how successful the 


pro ject leader felt the project was overa:l (56), and how well 
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the project leader thought the original objectives for the 
project had been satisfied (55). In other words, users and 
project leaders agreed very well on the relative success of 

a project, but implementation problems were a factor in the 
users' evaluations while they were not a factor in the project 
leader's evaluation of project success. 

Another very interesting aspect of user satisfaction 
came to light in looking at the relationships among the 
various criteria. It was felt that users might have been 
dissatisfied with projects which took too long to complete; 
that they disliked waiting for something they felt they needed. 
To get at this question, the individual projects that had 
the worst performance on time were checked for user satisfaction 
scores. It was already known that these two criteria were 
not significantly related for the whole study, but it was felt 
that looking at the extremes might reveal something that was 
hidden in the sample statistics. The surprising result of 
this investigation was that the project ranked last on time 
success was ranked third on user satisfaction, and the project 
ranked next to last on time success was ranked second on user 
satisfaction. In other words, two of the top three projects 
in terms of user satisfaction were the two bottom projects 
on time success. Upon closer review, what appeared to be 
underlying this finding was the evolutionary nature of the 


two projects in question. Both projects involved rather 
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complex models which were developed over a fairly long period 
of trial and error. Rather than taking a set of user stated 
desires or needs, and developing some programs to satisfy then, 
the systems staffs concerned worked very closely with the users 
in evolving those products which best fit the users' needs. 
There seemed to have been tacit recognition that the users 
would learn as they went, that they would become more sophis- 
ticated as they worked with outputs at successive stages of 
development, and would, therefore, not be satisfied with a one- 
shot development effort. Although the users were highly 
satisfied with what they ultimately received from the projects, 
this evolutionary approach took much longer than had originally 
been anticipated. 

Based on the above findings, an effort was made to 
discover other evidence in the data which supported what has 
been called the evolutionary approach to project development. 
It was hypothesized that projects which had not been developed 
with such an evolutionary strategy would reflect the following 
caaracteristics: 

Users who were originally satisfied with project 
results at the time the projects were completed 
might now be less satisfied with what they were 
receiving. This shift in satisfaction over time 
would have resulted from a user learning process 


which was not matched by any enhancements in the 
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information system outputs they were receiving. 

In other words, as the users became more sophis- 
ticated through working with the products of the 
projects in question, their information needs 
would shift. If the information system products 
did not shift with this increasing sophistication, 
the users would now be less satisfied with the 
projects than when they were first implemented. 

To test the above hypothesis, two separate components 
of the user satisfaction variable were analyzed. These 
components represented a measure of how well the user felt the 
original project objectives had been met, and a measure of how 
satisfied the user now was with the products of the project. 

It was already known that these two components of overall user 
satisfaction were highly correlated. However, it was not known 
if these two factors were scored at about the same level on a 
project by project basis. To get at this latter question, the 
mean value and the standard deviation were computed for each 
factor. These statistics were then used in testing the differ- 
ence between the two means to determine its significance. The 
mean value for the factor measuring how well original objectives 
were reportedly met was 39.2 with a standard deviation of 9.8. 
The mean and standard deviation for the reported current satis- 
faction factor were 33.5 and 10.3, respectively. The difference 


between these two means was 5.7, which, when tested by the t 
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statistic with 38 degrees of freedom, was significant beyond 
the .10 level for a twoetailed test. 

Although the t test described above was retrospective, 
and, therefore, not generalizable beyond the sample, it was 
interpreted as an indication that the users in the study did shift 
their perceived information requirements fairly quickly after MIS 
project implementation. While users may have scored a project 
very high on how well it served their information needs when 
first implemented, they tended to score present satisfaction at 
a lower level. This interpretation would seem to be a good 
argument for the evolutionary approach to MIS projects since 
such projects have been defined as management-decision oriented, 

Finally, with respect to user satisfaction, Appendix G 
contains comments by users relative to what they believed should 
have been done to improve the projects in the study, and what 
they view present shortcomings to be, These comments are not 
direct quotes; rather, they represent the notes taken by the 
researcher as the users responded to an open-ended question in 
the questionnaire (Appendix B, page 222). 

Computer Operations Problems (Var. 59). As reflected 
in Table 23, the measure of computer operations success used in 
the study was related to a secondary measure of such success, 
namely, the operations cost of a project. This relationship 
indicates that projects which were implemented most smoothly 


by computer operations were also the most successful in terms 
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statistic with 38 degrees of freedom, was significant beyond 
the .10 level for a twoetailed test. 

Although the t test described above was retrospective, 
and, therefore, not generalizable beyond the sample, it was 
interpreted as an indication that the users in the study did shift 
their perceived information requirements fairly quickly after MIS 
project implementation. While users may have scored a project 
very high on how well it served their information needs when 
first implemented, they tended to score present satisfaction at 
a lower level. This interpretation would seem to be a good 
argument for the evolutionary approach to MIS projects since 
such projects have been defined as management-decision oriented, 

Finally, with respect to user satisfaction, Appendix G 
contains comments by users relative to what they believed should 
have been done to improve the projects in the study, and what 
they view present shortcomings to be. These comments are not 
direct quotes; rather, they represent the notes taken by the 
researcher as the users responded to an open-ended question in 
the questionnaire (Appendix B, page 222). 

Computer Operations Problems (Var. 59). As reflected 
in Table 23, the measure of computer operations success used in 
the study was related to a secondary measure of such success, 
namely, the operations cost of a project. This relationship 
indicates that projects which were implemented most smoothly 


by computer operations were also the most successful in terms 
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of computer operating costs. This relationship is not 
surprising, and would appear to be the underlying 
explanation of the relationship between user satisfaction 
and operations cost success shown in Table 22. It will be 
recalled that the relationship of user satisfaction to 


computer operations success was discussed earlier. 


TABLE 23 
Computer Operations Problems(Var. 59) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 


Computer Operations Cost 60 «420 2.078 038 


Further Comments Concerning the Hypothesis Variables 


Tables 24 through 37 contain relationships of the 
hypothesis variables to other nonecriterion variables for 
which data were collected in the study. Each relationship 
tabled will not be dealt with in detail. Rather, patterns 
that were detected, and findings of particular interest, will 
be dealt with. The discussions which follow are oriented 
around the hypothesis variables, and are presented in the 
same sequence in which these variables appeared in the 
earlier chapter dealing with the testing of the hypotheses. 


User Participation-User (Var. 46). User participation 


in project development, as perceived by user personnel, was 
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related to more separate variables than any other variable 
in the study, as shown in Table 24, However, several of these 
relationships tended to cluster together, revealing what 


appeared to be more basic underlying relationships. 


TABLE 24 
User Participation-User (Var. 46) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
User Satisfaction-PL 0 0644 IU 0004 
Project Success -PL 56 my 31 15) 2.887 004 
Satisfaction of Objectives-PL 55 TOOLS 2 OT 006 
Time Success -PL 49 - 906 gs jay | o 006 
Assets i 2417 1.904 058 
Specificity of User 
Requirements -PL 38 400 2.160 030 
Measurable Project Objectives 12 394 Zehsl.2 . 036 
Complexity 13 - 369 2.009 o 046 
User Participation=-PL 41 o 367 2.174 » 030 
Specificity of User 
Requirements -User 45 356 Zeer 034 
Hardware Investment/Sales 6 ~ 309 |e bo’ 076 
Implementation Problems-User 53 By a sine Vy 090 


Two or More Years Systems 
Experience 27 Be.353 1.984 048 





| 
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User participation was significantly related to 
four separate measures of how successful the project leader 
perceived the project to be (Variables 49, 55, 56, and 57). 

The underlying relationships here would appear to be the 
relationship of user participation to user satisfaction, and 
the relationship of user satisfaction to the project leader's 
perceptions of success. Ome interesting exception was the 
relationship of user participation to the project leader's 
perception of time success. It appeared that where perceived 
user participation was high, project leaders tended to discount 
the actual time performance in evaluating time success. This, 
in turn, implies that project leaders were more oriented to 
how the users viewed the projects than to the actual time 
spent compared to time estimated, This observation will be 
dealt with in more detail further on, 

Those users in organizations which were large in 
terms of assets, and which had high computer hardware to sales 
ratios, felt that they participated more in project development 
efforts. This probably reflected a stage of development in 
computer usage which the larger, more heavily computer oriented 
organizations had reached, 

With respect to the reported specificity of project 
requirements, and the reported clarity of project objectives, 
user participation was perceived to be higher when the measures 


of these variables were higher. It was difficult to determine 
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precisely what was behind these relationships, but it seems 
that users felt they participated to a greater extent where 
projects were well defined and objectives were clearly stated 
from the beginning. The specificity and clarity of objectives 
would indicate that users knew what they wanted and were in a 
position to actively work with the systems staffs in getting 
it. Where objectives were vague, and the users weren't really 
certain what they were after, the information systems staffs 
picked up the ball and did more or less as they saw fit. 
This is not to impugn the information systems staffs. Rather, 
given somewhat vague requirements, they had to fill in the 
gaps themselves to get something running, relegating the users 
to question-answerers when the systems people got stuck. The 
important implication in these conclusions would seem to be the 
necessity for clear objectives and specific information require- 
ments from users if the users want to be heavily involved in 
the development effort, thus providing greater assurance that 
the project results will satisfy their information needs. 
Perceived user participation was also related to 
having fewer implementation problems. This logically would 
follow from closer user involvement during project development, 
thereby providing the users with a more precise understanding 
of what was to be implemented and possible attendant problems, 
Finally, one relationship shown in Table 24 which 


may appear somewhat odd was the one between perceived user 
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participation and the systems experience of the project staff. 
Being a negative relationship, one might conclude that staff 
personnel with greater systems experience did not permit much 
user participation. What actually seems to have been the case, 
however, was that user members of project teams, having little, 
if any, systems experience, pulled the systems experience 
measure down. Since having user personnel on project teams 
generally contributed to higher perceptions of user involve- 
ment, the negative relationship between participation and 
systems experience resulted. 

Measurable Project Objectives (Var. 12). There were 
two main clusters of variables related to the reported clarity 
of project objectives. The first cluster, consisting of 
Variables 1, 2, 3, and 6 in Table 25, represent organization 
and computer installation size. The larger organizations, and 
those most heavily committed to computerized information process- 
ing, tended to have the most specific and measurable project 
objectives. This, again, would seem to indicate a more advanced 
stage of computer usage. 

The second cluster of variables (14, 27, 39, 41, and 
46) seems to represent the effects of clear project objectives 
on other aspects of project development. Where project object- 
ives were reported to have been clearly stated from the beginning 
of a project, the users appeared to be more involved in all 


subsequent project efforts, a point made earlier. Project teams 
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were more common, top level user management took a more active 
interest in what was happening, and user participation, as 
perceived by both the project leaders and users, was higher. 
All of these factors combined to present a picture of users 
knowing what they wanted and actively participating with the 
systems staff in getting it. In addition, where project object- 
ives were reported to have been clearly stated in the beginning, 
there were fewer course changes during project development, as 
indicated by the negative relationship between objectives and 
the measure of project change requests honored by the project 
staff (Var. 40). 
TABLE 25 
Measurable Project Objectives (Var. 12) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Project Team 14 - 580 2.320 0020 
Assets (14 projects) l 2574 2.507 o012 
Emp loyees 3 2437 eI 0020 
Number on Project 17 «400 296b38 032 
User Participation-User 46 0394 2.112 036 
Sales (18 projects) Z 386 1.921 0054 
User Management Interest 39 2347 1.806 0072 
User Participation-PL 41 344 1.830 068 
Hardware Investment/Sales 6 312 1.660 096 
Two or More Years Systems 
Experience yay -.400 2.146 Uo 


Changed as Requested 40 -.405 12922 0054 
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An interesting exception to the above general 
conclusions was the case where project objectives were scored 
high, but the specific user requirements were scored low. There 
were three projects in this category. For these three projects, 
the original objectives were reported to have been quantified 
and clear. However, while the users apparently knew what they 
wanted to achieve, they didn't appear able to define the 
information content or format they needed to achieve it. In 
essence, project development seems to have begun before the 
users had analyzed their own information requirements well 
enough to define them clearly to the systems staff. The result 
in all three cases was considerable frustration among all 
concerned, and very poor time performance on the projects 
involved, The point of all this is that specific user require- 
ments did not automatically follow from well specified object- 
ives, and where they did not, considerable difficulties were 
encountered in project development. The fact that users were 
able to state clearly what they wanted to achieve did not mean 
that they understood their own information-decision environments 
well enough to plot a course to get there. 

Project Team (Var. 14). It was expected that the use 
of a project team, composed of user representatives and informa- 
tion systems staff, would enhance the level of user participation 
in project development. This expectation was confirmed when 


user participation was seen through the eyes of the project leader. 
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Where there were teams, consisting in part of user personnel, 
project leaders felt that objectives were clear, user require- 
ments were more specifically stated, and user participation 


was quite high (see Table 26). 


TABLE 26 
Project Team (Var. 14) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Specificity of User 
Requirements 38 750 3032 20027 
User Participation-PL 41 -660 22603 009 
Measurable Project Objectives 12 - 580 2432.0 0020 
Number on Project 17 - 900 1.964 050 
Two or More Years Systems 
Experience 27 -./760 3.018 004 


However, the measure of user participation representing the 
users' perceptions was not significantly related to the existence 
of a project team. This surprising finding for the whole sample 
led to a detailed review of each project in an effort to explain 
the divergence of the users' and the project leader's views of 
ostensibly the same factor, user participation. This detailed 
review revealed the following: 

1. The three highest ranked projects on user 

participation (users' view) were all 


developed by project teams. 
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2. The two lowest ranked projects on user 
participation (users' view) were not 
developed by project teams. 
3, Four projects where teams were used, but 
which had low ranks on user participation 
(users' view), seemed to reflect consider- 
able disagreement among individual user 
respondents on how much they, as users, 
participated in project development. 
To explore item 3 above further, eighteen of the questionnaires 
were analyzed in terms of individual user responses to the 
four questions that were aggregated in arriving at the user 
participation scores for these projects. Only eighteen projects 
were analyzed, since for two projects there was only one user, 
and no disagreement on the separate questions was possible. For 
each of the eighteen projects analyzed, an "index of disagreement" 
was computed as follows: 
For each of the four questions comprising the 
composite variable, user participation, the 
difference between two separate user responses 
was determined by merely subtracting the low 
response from the high response on the fifty 
point scale. These four differences were 
then summed together to get an index of 


disagreement for the project. For example, 
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if one user responded to the question 

"What was the degree of your organization's 

active participation throughout the evolu- 

tion of this project?" with a scale value 

of forty, and the other user responded 

with a scale value of twenty, the difference 

for that one question was twenty. That 

difference was then added to the remaining 

three differences computed in the same 

manner to arrive at the index value for the 

project. 

The mean value of the index of disagreement for all 
eighteen projects was 45.6. When the index of disagreement 
value for each of the four projects mentioned in item 3 above 
was compared to this mean index value, it was found that these 
four projects had the highest index of disagreement scores in 
the sample, with two at 70, one at 100, and one at 120. Further, 
all four of these projects had one thing in common: each was a 
case where one of the user personnel interviewed had actually 
worked on the project team and the other user had not. As 
would be expected, the individuals who had actually been on 
project teams scored user participation consistently higher 
than did the users who had not been on the project teams. 

What the above findings would seem to indicate is 


that users disagreed among themselves concerning how much they 
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participated in a project. While the individuals who were 
actually on the team felt that participation was high, other 
users often felt they had very little to do with the project. 
Where the objective of a project was to provide management 
information to several people in a user area, in fact, only 
those who actually were on the project team ended up getting 
what they wanted. Thus, while having a project team did seem 
to facilitate users feeling they participated to a larger 
degree, a team did not assure such feelings of participation. 
This situation was particularly obvious where the user represen- 
tatives on a team were staff personnel from the user area, but 
the actual recipients of the results of the project were other 
individuals who had not been very much involved in project 
development. 

With respect to the perceptions of project leaders 
concerning user participation, no distinctions were apparently 
made among the kinds of users who were on project teams. The 
mere presence of user representatives led to perceptions of 
high user participation, thus the high relationship between 
Variables 14 and 41 in Table 26, 

Another finding of interest relative to the use of 
project teams, was that user satisfaction with results was 
lower where the user members of project teams were not the 
ones responsible for implementing the completed projects, 


nor for maintaining on-going liaison with the information 
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systems staffs. There were two projects which fell in this 
category. Whereas the mean value of user satisfaction (Var. 54) 
for all projects in the sample was 175, the mean value of user 
satisfaction for the two projects cited was 122.5. Both of these 
projects were in the bottom half of the user satisfaction rank- 
ing, with ranks of 12 and 15. 

Finally, the fact that user representatives were 
full-time or part-time on project teams seems to have made no 
difference on any criterion. Having part-time or full-time 
user members on a project team appeared to depend more on the 
nature of the project and the structure of the organization 
than anything else. 

As would be expected, Table 26 shows that there 
tended to be more people involved in projects where teams were 
used. Also, the level of systems experience was lower where 
teams were used, since users, who had little or no systems 
experience, pulled the measure of systems experience down. 

Level of Information Systems Manager (Var. 9). For 
the seven organizations in the sample which provided asset 
data, the larger organizations, in terms of assets, tended to 
place the top information systems executive closer to the top 
operating executive of the organization. This is reflected in 
Table 27 as a negative relationship, since a lower value for 
Variable 9 meant fewer hierarchical levels between the informa- 


tion systems executive and the top operating executive. 
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TABLE 27 
Level of Information Systems Manager (Var. 9) 


(Twortailed significance at or beyond .10 only) 


Var. Tau S* P(s 


Assets (Ns7) 1 -./96 -13.0 007 


* Since asset data were available for only 7 organizations, the 
normal deviate was replaced by the actual computed value of S. 


The probability level was determined from Sillitto (1947). 


Two or More Years Systems Experience (Var. 27). All 


of the relationships shown in Table 28 would seem to reflect, 
essentially, the degree of user involvement in the projects in 
the study. There are no relationships there which are not 
consistent with the discussions already presented for other 
variables. Clear objectives, specific user requirements, 

user participation, the number of people on the project, and 
the use of a project team have all been shown to be related 

to each other. Since what was posited as underlying all of 
these relationships was user involvement, it is not surprising 
that 4ll of the variables shown in Table 28 were negatively 
related to the systems experience measure, inasmuch as that 
measure was lower the more users were directly involved in 


project development. 
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TABLE 28 
Two or More Years Systems Experience * (Var. 27) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 

Project Team 14 -. 760 3.018 003 
Specificity of User 

Requirements -PL 38 -.450 2.453 014 


Measurable Project Objectives 12 -.400 2.146 032 


Number on Project 17 ~-.394 20011 0022 
User Participation-PL 41 -.361 2.141 032 
User Participation-User 46 Be-.353 1.984 048 
User Satisfaction-PL 57 -.319 1a 00 084 


*Since all values for tau are negative in this table, the 
entries are from most negative (strongest relationship) to 


least negative (weakest relationship). 


Documentation Standards (Var. 35). Although the 
reported existence of formal documentation standards is shown 
to be related to three variables in Table 29, great caution 
must be exercised in making any generalizations about these 
relationships. Only four projects in the study had no documen- 
tation standards, and two of these projects were in one 
organization. This organization was the smallest in the sample 
in terms of sales, and next to the smallest in number of 


employees. Further, all of the project staff members for both 
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of these projects held college degrees, A third project for 
which no formal documentation standards applied was in the 
smallest organization in the sample in terms of employees, and 
this organization was next to the smallest in sales. This 
third project, a sophisticated modeling application, was also 
developed by a team composed entirely of college graduates. 
These factors, together with the nature of the tau statistic 
computation, explain the relationships found, Therefore, 


generalizing in this case would be very hazardous. 


TABLE 29 
Documentation Standards (Var. 35) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Sales (18 projects) 2 543 2.559 ~010 
Emp Lloyees 3 520 2.149 032 
College Degree 30 -.490 2.304 022 


Project Control (Var. 42). The primary conclusion one 
draws when looking at the relationships between the project 
control measure and other variables in the study is that 
reported project control efforts were, essentially, ineffective, 
perhaps even dysfunctional. It has already been shown in Tables 
16-19 that reported project control efforts were not related to 


any criterion of success. This would seem to indicate that 
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project status information, which was ostensibly collected for 
control purposes, was not, in fact, used to exercise control, 
at least over the project itself. It is worth noting that of 
the twenty projects in the study, in only one case did 

project status reporting result in additional resources being 
committed to the project. That one project was completed on 
time, In other projects, however, it appeared that project 
status information was collected as "nice to know", or, at 
best, was used to keep others outside the information systems 
function posted on when they would be affected by conversion 
or implementation activities. This is not to say that such 
overall organizational awareness is not desirable nor necessary, 
But awareness and positive steps to influence project develop- 
ment rate are not synonymous. 

Analysis of Table 30 reveals some further interesting 
aspects of project control. Although reported project control 
was not related to time success, it was quite strongly related 
to the project leader's perception of time success. What this 
relationship seems to reflect was the tendency of the project 
leader to feel he had done his best where he was constantly 
maintaining close tabs on project status. He didn't feel the 
project had really been so bad in terms of time success, and 
tended to blame overruns on poor estimates, although in only 
one case did a project leader feel the original time estimate 
for the project had been unrealistic! In fact, two projects, 


which exceeded time estimates by 600% and 9007, were rated 
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average in terms of time success, and three projects, which 
exceeded time estimates by 135%, 146% and 180%, were all rated 


above average on time success. 


TABLE 30 
Project Control (Var. 42) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Time Success-PL 49 - 540 2.861 004 
Originator ll -.322 l 22 084 
Quality of Documentation 36 Be. 336 1.650 . 100 
Changed as Requested 40 -.345 1.683 e932 


It appeared that project leaders felt an implicit 
pressure from tight project reporting requirements, but they 
felt helpless in doing anything about improving poor perform- 
ance beyond cutting corners in such areas as documentation and 
preparation for implementation. The negative relationship 
between reported project control efforts and the perceived 
quality of documentation would seem to lend support to this 
notion, as would the negative relationship between reported 
project control and perceived implementation problems. This 
latter relationship is not tabled, as it was not significant 
at the .10 level, but it was fairly strong, with a tau of -.30/7 


and a normal deviate of 1.638. 
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Another indication of the pressure felt by the 
project leader from project control schemes was the reported 
reluctance to make changes as development progressed. This 
was not necessarily a bad thing, but the pressure to get done 
as soon as possible seems to have prevented changes needed to 
provide users with more acceptable outputs in some cases. 
Although the reluctance to make desired changes did not seem 
to contribute to being on time with projects, it probably did 
prevent several of the projects from being more over estimate 
than they already were. 

Finally, as shown in Table 30, those projects 
initiated by the information systems staffs appeared to be 
subject to more stringent control requirements than those. 
initiated by users. It would seem that the planning process 
for project development was more explicit when a project came 
from the information systems staff, resulting in more specific 
provisions for control of the project during its development. 
This observation opens up some very interesting possibilities 
with respect to kinds of information systems projects. These 
possibilities will be dealt with in Chapter VIII. 

Turnover (Var. 29). It was pointed out earlier that 
the turnover of project personnel could not be fairly evaluated 
because only three projects in the study experienced any turn- 
over at all. The same caution must be observed in considering 


the relationship of turnover to implementation problems, as 
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viewed by the project leader, shown in Table 31. While the 
turnover that did occur on projects in the sample appeared 

to create real difficulties for the project leaders of those 
projects, it would be risky to generalize beyond those projects. 
The discussion of just what implementation problems seemed to 
consist of in the eyes of project leaders will be dealt with in 


the next chapter. 


TABLE 31 
Turnover (Var. 29) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 


Implementation Problems-PL 58 -.322 22d 020 


Originator (Var. 11). The relationship of reported 
project control efforts to origin of a project has already 
been discussed. The relationship of project origin to implemen- 
tation problems, as viewed by the user, shown in Table 32, 
indicates that users felt they had less difficulty implementing 
projects they initiated than those initiated by higher level 
management or information systems staffs. This finding is not 
at all surprising, and fits the patterns of relationships 
already discussed. 

One finding that was surprising, however, was the 


absence of a significant relationship between origin of a 
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project and the users’ perception of participation in project 
development. Although these two variables were related, with 
a tau of .29 and a normal deviate of 1.493, this relationship 


failed to reach significance at the .10 level. 


TABLE 32 
Originator (Var. 11) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Implementation Problems-User 53 330 1,709 «088 
Project Control 42 «322 1.722 084 


A closer investigation of the sample data revealed 
what would appear to explain this situation. Two of the thirteen 
projects initiated by users had very low values for perceived 
user participation, 120 in both cases. This value was well 
below the overall mean for user participation, 152.6, and was 
far below the mean user participation value of 164 for those 
thirteen projects which were originated by users. The two 
projects in question, although originated by users, were 
developed, for the most part, without further substantive user 
participation. In one case, this appeared to be the result of 
the users not wanting to be bothered; and in the other case, the 
project leader simply did not work with the users, choosing to 


proceed on his own. It is worth noting that these two projects 
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were both very low in the ranking of user satisfaction, with 
one project ranked 17, and the other project ranked 18. 

Based on the above findings, it appears that the 
fact a user originated a project did not guarantee his partici- 
pation in its development. However, the results were not very 
satisfying to the user where participation did not follow 
from project initiation. 

Centralization of MIS Activities (Var. 8). This 
variable was not adequately tested in the present study. 

Two or More Years Organization Experience (Var. 34). 
There were few findings with respect to organization experience 
of the project staff which were of any significance. As might 
be expected, Table 33 shows that larger organizations, in 
terms of numbers of employees, had project staffs with less 
organization experience. This might be interpreted as an 
indication that more fluid personnel situations existed in 
larger organizations. Table 33 also shows that the education 
level of the project staff was related to organization experience. 
This finding would seem to run counter to the popular notion 
that people involved in systems work, and who are highly 
educated, have high job turnover rates. At least for this 
sample, those who were more highly educated tended to have 
been in their current organizations for at least two years. 
Caution must be exercised in making too much of the above 


finding concerning turnover and education, however. The 
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measures used were not direct individual measures of job 
longevity and education level. Rather, they were mean values 
and proportions, which were felt to be the best types of 
measures for the primary tasks in this study, but which were 
not the best measures for an analysis of turnover as it 


relates to education. 


TABLE 33 
Two or More Years Organization Experience (Var. 34) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
College Degree 30 B.375 2.132 034 
Mean Years Formal Education 32 0372 2.046 040 
Emp loyees 3 -.411 2.401 016 


High Level Programming Language (Var. 43). The 


relationship between the use of a higher level programming 
language and the quality of project documentation, shown in 
Table 34, reflects primarily the views of project leaders 

that the use of COBOL resulted in good program documentation. 
Since the perceived quality of program documentation appeared 

to be a key element in evaluating overall project documentation, 


this finding is not surprising. 
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TABLE 34 
High Level Programming Language (Var. 43) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 


Quality of Documentation 36 -300 1.780 076 


Mean Years Formal Education (Var. 32). The relationship 
found between project personnel education levels and organization 
experience has been dealt with to some extent in the discussion 
of the organization experience variable. As shown in Table 35, 
there were four other variables significantly related to project 
staff education besides organization experience. Looking at ail 
five of these variables together, one finds a pattern that seems 
to explain their relationships to education levels of project 
staffs. The smaller organizations in the sample, in terms of 
numbers of employees, tended to have smaller project staffs whose 
members had high education levels, and who had been employed in 
their respective organizations for over two years. Further, 
smaller project staffs tended to have combination analyst/ 
programmers. Finally, users generally perceived implementation 
problems to be the least where combination analyst/programmers 
developed projects (see Table 36). The above pattern was partic- 
ularly pronounced where applications dealing with mathematical 


models were involved. This does not mean that smaller organizations 
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had the most successful projects or the projects with the least 
implementation problems. Rather, the interrelations of the 
several variables discussed provide a possible explanation 


for findings relative to the education level of project personnel. 


TABLE 35 
Mean Years Formal Education (Var. 32) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 

Combination Analyst/Programmer 23 440 1.668 096 
Two or More Years Organization 

Experience 34 so72 2.046 040 

Implementation Problems-User 53 ~ 336 1.884 - 060 

Number on Project 17 -.330 1.821 - 068 

Emp Loyees 3 -.378 2.073 038 

TABLE 36 


Combination Analyst/Programmer (Var. 23) 
(Twostailed significance at or beyond .10 only) 


Var. Tau Z(3s P(Z 
College Degree 30 590 2.238 026 
Implementation Problems-User 53 - 530 2.029 042 
Mean Years Formal Education 32 440 1.668 096 
Man Months 22 -.900 1.868 062 


Number on Project 17 -.610 2.306 e022 
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Combination Analyst/Programmer (Var. 23). The 


finding in the study that project staffs comprised of 
combination analyst/programmers had higher levels of education 
has already been mentioned in the preceding discussion of the 
primary measure of education level. Table 36 also shows that 
there were higher proportions of college degrees on project 
staffs composed of combination analyst/programmers. This is 
merely one more way of looking at the same basic relationship. 

The tendency for users to feel implementation 
problems were less with combination analyst/programmers has 
also been cited on several occasions. As was pointed out 
before, users seemed to feel frustrated and somewhat dissatisfied 
with implementation efforts where they had to deal with differ- 
ent people from the information systems staff, depending on the 
nature of the problems that came up. The ability to deal with 
one person who understood all aspects of the project, and who 
could respond to problems with action, was often mentioned by 
users as a very strong factor in their satisfaction with project 
results. It appears that combination analyst/programmers were 
better able to meet users' desires in this respect than were 
separate analystsor programmers. 

Finally, combination analyst/programmers generally 
meant fewer people on a project, and the projects they worked 
on took fewer man months. This latter relationship is open 


to interpretation. It could be that since there were fewer 
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people working on projects where combination analyst/progranmmers 
were used, the man months being less just reflected that under- 
lying relationship. On the other hand, it could be that there 
was less lost time and motion where combination analyst/ 
programmers were used, thus requiring fewer man months to 
complete a project. While this latter argument is appealing 
from a logical standpoint, it is hard to support in view of 

the fact that, as shown earlier in the discussion of criteria- 
hypotheses relationships, project staffs composed of combination 
analyst/programmers performed rather poorly on the time success 
criterion. Another possible explanation might be that combina- 
tion analyst/programmers were used on smaller, less complex 
projects. This position is also difficult to defend, however, 
since there was little relationship between the measure of 
complexity (Var. 13) and the use of combination analyst/programmers. 
Perhaps the question could have been resolved had there been a 
comparable, objective measure of project complexity which was 
applied equally to all projects by the researcher. In the 
absence of such a measure, no satisfactory explanation could 

be found by the researcher for the relationship between using 
combination analyst/programmers and man months spent on a 
project. The whole question of combination versus separate 
analyst/programmers would seem to be a very fruitful one for 


further research, 
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Systems Staff/Total Employees (Var. 61). The 


measure of relative size of sample organizations’ systems 
staffs was not significantly related to any other independent 
variable with which it was cross-tabulated. 

Hardware Investment/Sales (Var. 6). Organizations 
in the sample with the highest asset valuations also tended 
to have the highest ratios of computing hardware investment 
to sales for the past fiscal year. This relationship was 
computed for only the seven organizations which provided 


asset data, however (see Table 37). 


TABLE 37 
Hardware Investment/Sales (Var. 6) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Assets (N57) 1 784 $216.0* .02 
Measurable Project Objectives 12 312 1.660 . 096 
User Participation-User 46 . 309 1.772 076 





* Since asset data were available for only 7 organizations, 
the normal deviate was replaced by the actual computed value 
of S. The probability level was determined from Kendall (1962). 





The hardware investment/sales ratio was also related 
to the reported clarity of project objectives, and to user 


perceptions of participation in project development. It has 
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already been shown that perceived user participation was related 
to reported clarity of project objectives (Table 24). As was 
pointed out in the discussion of relationships shown in Table 
24, a high hardware/sales ratio was construed to represent a 
strong commitment by an organization to computerized processing 
of information for decision making. In such an environment, 

it appears that clear objectives were required before a project 
was initiated, and users were more involved in the planning for 


their own information systems, 
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CHAPTER VIT 


OTHER RELATIONSHIPS OF INTEREST 


Although the author's primary intent in this thesis 
has been to report the results of conducting tests of the 
hypotheses posited for the study, the nature of the data 
collected has made it possible to investigate and report 
certain other relationships of relevance and interest. Some 
of these "other" relationships have already been discussed 
in the last two sections of Chapter VI. This chapter focuses 
mainly on those findings which provide insight into the ways 


project leaders viewed their respective projects. 


Project Leaders' Views on Project Success 


Tables 38-41 provide a picture of the way project 
leaders viewed project success in terms of time, cost, user 
satisfaction, and for the project overall. No attempt has been 
made to discuss every relationship shown in all four tables. 
Rather, what appeared to be meaningful patterns of relationships 
have been dealt with. 

First, although project leaders’ evaluations of 
time and cost success were related to the actual measures of 
time success and cost success respectively, the most prominent 
factor underlying the overall evaluations of success appeared 
to be how satisfied the project leaders perceived the users 


were with the project results. To support this contention, 
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consider the results in Tables 40 and 41. When asked to rate their 
projects on how successful they were overall, the project leaders’ 
responses were related to a cluster of other variables which repre- 
sented, essentially, user satisfaction. The only exception was 


the relationship between overall project success and time success. 


TABLE 38 
Time Success-PL (Var. 49) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Project Control 42 ~ 540 2.861 004 
User Participation-User 46 « 506 Lebar o 006 
Project Success-PL 56 487 2.578 -010 
Cost Success-PL 52 B.447 2.335 020 
Specificity of User 
Requirements -User 45 394 2.114 2034 
User Satisfaction-PL 57 B, 387 2.006 044 
Elapsed Months 21 -.369 1.963 050 
Actual Time/Estimated Time 48 -,437 2.328 020 


br 


TABLE 39 
Cost Success-=PL (Var. 52) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Time Success -PL 49 B.447 2.335 ~020 
Number on Project 17 -.319 1.707 » 088 


Actual Cost/Estimated Cost 51 ~-.644 3.444 .- 0006 
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TABLE 40 


Project Success-©PL (Var. 56) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z2(s P(Z 

User Satisfaction-PL 57 713 35837 0002 
User Satisfaction-User 54 607 3.124 002 

Implementation Problems-User 53 © 553 2.920 2 004 

User Participation-User 46 553 2.887 ~004 

Time Success=PL 49 2487 2.578 010 

Satisfaction of Objectives-PL 55 B.407 1.979 048 

Specificity of User 

Requirements -PL 38 - 367 1.937 s0s2 
TABLE 41 


User Satisfaction-PL (Var. 47) 


(Twoetailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 

Project Success-PL 56 anal 3.837 ~0002 
User Satisfaction-User 54 656 3.548 ~0004 
User Participation-User 46 0644 3.530 0004 
Implementation Problems-User 53 519 2.871 004 
Satisfaction of Objectives=-PL 55 0447 2.469 014 
Specificity of User 

Requirements -PL 38 B.409 2.116 036 
User Participation-PL 41 0394 Zeloo » 032 
Time Success -PL 49 B.387 2.006 044 
Two or More Years Systems 

Experience 27 -.319 1.730 084 
Man Months 22 -,350 1.880 060 
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While it appears that project leaders did give some 
consideration to time success, as they had evaluated it, in 
arriving at an overall rating of project success, that relation- 
ship, too, may reflect a "user satisfaction" bias. This last 
statement follows from the findings in Table 38, Time success, 
as viewed by the project leader, was related to reported project 
control, which has already been discussed; to the actual time 
success measure; and to the project leader's rating of cost 
success. However, it was also related to user satisfaction, 
as viewed by the project leader, perceived user participation, 
and the reported specificity of user requirements. These 
last mentioned relationships could reflect the tendency of 
project leaders to view time success in the context of how 
satisfied they felt users were with project results. This 
possibility is supported by the fact that the two lowest 
ranked projects (19 and 20) on the user satisfaction criterion 
were rated at the lowest level (very poor) on time success 
(Var. 49) by the project leaders. However, these two projects 
were ranked 13 and 14 on the actual measure of time success 
(Var. 48). 

On the other hand, it may be that project leaders 
did rate overall project success with consideration given to 
their perceptions of both time success and user satisfaction. 

If this was the case, and the two projects cited in the previous 


paragraph were merely chance occurrences, the user satisfaction 
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relationship to perceived time success could be explained by 
the common relationship of both of those variables to overall 
project success (Var. 56). 

With respect to the strong relationship between the 
project leader's perceptions of time success and cost success, 
it would appear that a project leader response set was partially 
responsible for this relationship. Although, as Table 15 
shows, the actual measures of time success and cost success 
were related, that relationship was not significant at the 
~-10 level. This led to the conclusion that project leaders 
tended to rate cost success and time success at about the 
game level, regardless of how similar the actual time 
success and cost success values were. 

Perhaps the most important point to be made concerning 
the relationships shown in Tables 40 and 41 is how accurately 
the project leaders perceived user satisfaction with their 
projects. It was evident that in nearly all cases, project 
leaders were very well attuned to how the users viewed project 
results. It was also apparent, as has been discussed above, 
that those user attitudes were key determinants in how 


successful project leaders viewed their projects to be, 


Different Views of Implementation Problems 


One interesting aspect of project leaders’ perceptions 
concerning user satisfaction was their awareness of implementa- 


tion problems, as viewed by the users. As Tables 40 and 41 
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show, the users' evaluation of implementation problems was very 
strongly related to how well satisfied the project leader 
perceived the users to be, and, in turn, to how successful the 
project leader felt the overall project was. It has already 
been shown (Table 22) that users considered ease or difficulty 
of project implementation to be a very significant component 

of overall satisfaction with a project. 

However, as Tables 42 and 43 show, project leaders 
viewed implementation problems very differently than users did. 
Not only were the two views of implementation problems different, 
project leaders did not even consider implementation problems 
to be related to any criterion of success. 

Most of the relationships in Table 42 have been 
covered in earlier discussions of the several variables shown, 
go there is no need to repeat what has been said about then. 
However, taking all of the variables together, and considering 
the pattern, provides insight into what decreased implementation 
problems from the users' perspective. 

Implementation to users appears to have meant getting 
the project and its outputs incorporated into their ongoing 
operations as smoothly and effectively as possible. The object- 
ive of the user was to get an information system product which 
aided in decision making, but which also caused the least 
turbulence in the organization. Achieving this objective 


appeared to be tied to how prepared the user was for what he got. 
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TABLE 42 
Implementation Problems-User (Var. 53) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
User Satisfaction-User 54 720 4,192 0004 
Project Success-PL 56 553 22920 004 
Combination Analyst/Programmer 23 ~ 530 2.029 042 
User Satisfaction-PL 57 919 2.871 004 
Specificity of User 
Requirements -PL 38 419 2.290 #021 
Mean Years Formal Education 52 . 336 1.884 .060 
Quality of Documentation 36 #33 1.739 082 
Originator ll ~ 330 1.709 088 
User Participation-User 46 291 1.701 090 
Man Months 22 -.383 2. 27 026 
TABLE 43 


Implementation Problems-PL (Var. 58) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Quality of Documentation 36 533 2.869 004 
Turnover 29 -.322 29327 .020 


Complexity 13. B=. 352 1.793 .074 
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When he had originated a project, participated heavily in its 
development, and had good project documentation to work with, 
the user knew what was coming. Further, where he was able 

to deal with one individual who knew the whole picture when 
problems did arise, the user felt he was able to get those 
problems rectified quickly. 

Contrast this to how the project leader viewed imple- 
mentation problems. Although, as was mentioned earlier, the 
project leader was aware of the users’ view of implementation 
problems, he did not consider the same factors to be the 
components of implementation problems, with the exception of 
documentation. The project leader tended to view successful 
implementation as getting the programs running, the documenta- 
tion finished, and the procedural flows correct so that his 
system worked. Turnover, although very limited in the sample, 
was seen as standing in the way of easy implementation, as was 
pressure to rush a project to completion. Reported complexity 
of the project was also negatively related to the ease of 
implementation, but it was difficult to determine which way 
this relationship went. It could be that project leaders 
viewed as complex those projects which were difficult to 

1 although not significant at the.l0 level, implementation 
problems (Var. 58) was negatively related to project control 


(Var. 42) with a tau of -.307 and a normal deviate of 1.683, 
significant at the .102 level (two-tailed). 
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implement. Finally, where documentation was felt by project 
leaders to be good, implementation problems were felt to be less. 
Although not entirely accurate, one generalization 
which might best describe the differing views of implementation 
problems is that project leaders tended to view implementation 
up to operational cutover of a project, while the users looked 


at implementation as what happened after operational cutover. 


Project Leader's Perception of User Participation 


In general, the project leader felt there was high 
user participation if a project team consisting of some user 
representatives developed the project, and if he perceived top 
level user management had actively supported the project. 
Most of the relationships in Table 45 represent either these 
two basic variables or other variables already shown to be 
related to them. Further, the project leader tended to feel 
that the user had stated clear and specific project require- 
ments where there was a high level of user participation, as 
indicated by the fact that most of the variables shown in 
Tables 44 and 45 are the same. 

On the other hand, the users’ evaluation of the 
specificity of their original desires and requirements was 
related to a separate group of variables as shown in Table 46. 
This evaluation was not significantly related to the project 
leader's view of ostensibly the same factor, specificity of 


user requirements. It appeared that users felt they had to 
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be more specific in stating what they wanted in larger organiza- 
tions to get their projects accepted. It also appeared that 

the users were more accurate than project leaders in their 
appraisals of how clearly they stated exactly what they wanted 
from projects. Although not significant at the .10 level, 

the users’ view of specificity of their requirements (Var. 45) 
was related to the time success criterion (Var. 48) with a tau 
of -.272, a normal deviate of 1.586, and a probability of .114. 
Further, there was a significant relationship between Variable 
45 and both the elapsed months spent on the project, and the 


project leader's appraisal of time success. 


TABLE 44 
Specificity of User Requirements-PL (Var. 38) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Project Team 14 750 3.032 -0027 
User Man Months/ 

Total Man Months 15 262 07S ~0022 
User Participation-PL 41 562 3.036 0025 
Implementation Problems-User 53 ~419 2.290 021 
User Satisfaction-PL 57 B.409 2.116 036 
User Participation-User 46 - 400 2.160 030 
Project Success -=-PL 56 367 1.937 052 
User Management Interest 39 ~320 1.675 094 


Two or More Years Systems 
Experience 27 -.450 2.435 .014 
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TABLE 45 
User Participation-PL (Var. 41) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Project Team 14 -660 2.603 009 
Specificity of User 

Requirements-PL 38 562 3.036 -0025 
User Man Months/ 

Total Man Months 15 0259 3.2599 ~0014 
User Management Interest 39 453 26352 020 
User Satisfaction-PL 57 394 135 032 
User Participation-User 46 367 2.174 030 
Measurable Project Objectives 12 344 1,830 . 068 
Two or More Years Systems 

Experience a | -.361 2.141 032 

TABLE 46 


Specificity of User Requirements-User (Var. 45) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Employees 3 B.405 2.316 . 020 
Time Success-PL 49 0394 2.114 034 
User Participation-User 46 356 2.111 034 
Elapsed Months Zi) -.300 1.756 080 


College Degree 5 -.333 1.977 048 
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On the assumption that clear, specific, and detailed 
user requirements provided the project staff with a more concrete 
nucleus around which to build the project, time success should 
have been greater. The data would appear to support this 
contention. 

User participation (Var. 46) also appeared to be 
greater when users felt they had been very specific in stating 
their requirements. As was mentioned at an earlier point, 
where the users were able to state exactly what they wanted from 
a project, they apparently felt they had control of what was 
developed, and, therefore, that they had high participation. 

It may strike the reader as odd that there was no 
relationship between reported specificity of user requirements 
(Var. 45) and user satisfaction (Var. 54). However, a close 
investigation revealed three projects, termed "evolutionary", 
which were low on Variable 45, but high on user satisfaction 
(Var. 54). There were also three projects which were high on 
Variable 45, but very low on Variable 54, a result, apparently, 
of users not getting what they had specified as required. 

These several projects, in effect, "cancelled out" any relation- 
ship between specificity of user requirements (Var. 45) and 

user satisfaction (Var. 54) in the computation of the tau 
statistic. 

In summary, whereas the project leader appeared to 


base his evaluation of the specificity of user requirements 
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on the degree of user participation in the project, the users 
seemed to base the appraisal of their participation, in part, 

on how specifically they stated their requirements in the first 
place. There were cases, however, where users were not specific 
in stating their requirements, but felt they participated 
through an evolutionary process of trial and error over a 
relatively long period of project development. Finally, users 
and project leaders did not tend to agree on the degree of 
specificity in original project requirements, but the data 
suggest that perhaps the users’ appraisals were more accurate 
than those of project leaders. This last point, on the accuracy 
of appraisals, should be researched more thoroughly, however, 


before any firm conclusion is drawn. 


Quality of Documentation 


For the projects in the study, the perceived quality 
of documentation was not related to the presence or absence of 
documentation standards. Again, it is emphasized that only 
four projects were developed without documentation standards, 
so it would seem unreasonable to conclude that documentation 
tends to be as good without standards as with them. However, 
it did appear that the mere presence of standards was not enough 
to assure high quality documentation. 

As shown in Table 47, the perceived quality of project 
documentation wa: ~elated to several other variables. It has 


already been poiiu:2d out that both users and project leaders 
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perceived fewer implementation problems where documentation was 


of relatively high quality. This finding was not unexpected. 


TABLE 47 
Quality of Documentation (Var. 36) 


(Two-tailed significance at or beyond .10 only) 


Var. Tau Z(s P(Z 
Implementation Problems-PL 58 599 2.869 004 
Implementation Problems-User 53 333 1.739 082 
High Level Programming 
Language 43 - 300 1.780 076 
Man Months 22 -.327 1.663 096 
Project Control 42 Be.336 1.650 ~10 
Elapsed Months 21 -.340 1.736 082 


The relationship of perceived documentation quality 
to the use of higher level programming languages, essentially 
COBOL, has also been discussed. It appeared that most project 
leaders who used COBOL felt they had adequate documentation to 
allow reasonably easy program maintenance, which seems to have 
been their primary criterion for quality. 

Finally, the perceived quality of project documenta- 
tion was negatively related to the elapsed time on the project, 
the man months on the project, and reported project control. 


These rather weak relationships could be interpreted at least 
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two different ways. One interpretation would be that projects 
took longer because of poor quality documentation that occasioned 
delays, duplication, and general confusion. Another interpreta- 
tion, which seems to make some sense, would be that where project 
control schemes were more stringent, pressure built up on project 
leaders and staffs as the project took longer or more man 

months were expended. The response to the pressure was to cut 
corners on documentation, concentrating on getting something 
running as quickly as possible. If the latter possibility were 
true, the control schemes used could have been dysfunctional 

to long range project success. Most of the prescriptive litera- 
ture argues for both documentation standards, to assure high 
quality documentation, and project control techniques, to keep 
projects on target. They may not be compatible. This is a 

very interesting question which should be investigated by 


experimental means if possible. 





PART IV 


SUMMARY OF RESULTS, CONCLUSIONS, AND 


SUGGESTED FUTURE RESEARCH 
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CHAPTER VIII 


SUMMARY OF RESULTS 


Introduction 
In Chapter I the purpose of the research described in 
this thesis was phrased as the question: 


What organizational and procedural factors 
are correlates of success with MIS projects? 


In seeking answers to that question, the literature on MIS, 
project management, and information systems in general was 
reviewed. While few satisfactory answers were found in the 
literature review, a set of thirty-five hypotheses was 
developed which, it was hoped, could be tested. 

The list of hypotheses was pared down from thirty-five 
to sixteen after analyzing the responses of information systems 
professionals to a request to rank the thirty-five hypotheses, 
or factors, in terms of their importance to MIS project success. 
These sixteen hypotheses, along with four criteria of MIS project 
success, then became the nucleus of a questionnaire which was 
developed for interviewing people in organizations who had been 
involved with MIS projects. 

A successful pre-test of the questionnaire was followed 
by the selection of a study sample. Ten organizations from the 
Twin Cities, representing seven industries, were chosen for the 
study. Data were collected on two MIS projects in each of these 


organizations between October and mid-December, 1970. 
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The data collected were analyzed to determine what 
relationships existed among variables in the study. The primary 
form of statistical analysis was the computation of Kendall's 
rank correlation statistic, tau, for every possible relationship 
between any two of the sixty-one variables identified in the 
study. 

The results of the data analysis have been discussed 
in detail in Chapters VI and VII. This chapter is devoted to 
summarizing the findings of the study. The final chapter, 
Chapter IX, will be devoted to drawing conclusions from the 
results, and to suggesting some possible future research 


directions based on what has been found here, 


Independence of Criteria of Success 


Four criteria of MIS project success were posited and 
used in this study: time success, cost success, user satisfac- 
tion, and computer operations success. These four criteria were 
found to be reasonably independent measures of MIS project 
success. The closest relationship found between any two criteria 
was that between user satisfaction and computer operations success. 
This would seem to be a logical relationship. Where a project 
impacts on computer operations in such a way that scheduling 
and running problems develop, the users probably feel such 
problems through inadequate response and service from the 


information systems function. 
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Findings: Tests of Hypotheses 


Hypotheses Related to Criteria of Success 


Of the sixteen hypotheses which were selected to be 
tested in the present study, one was not effectively tested 
by the sample data, ten were found to be significantly related 
to at least one criterion of success, and five were found to 
be unrelated to any criterion of success. The hypothesis 
which was not tested, and about which no conclusions could be 
drawn, was the degree of relationship between centralization 
of the information systems function and MIS project success. 

The ten hypotheses which were tested and found to 
be related to project success are shown in Table 48. For 
each entry, the rank of the hypothesis, in terms of its importance 
to project success, from’ the original ranking of thirty-five 
hypotheses is shown first; next is shown the hypothesis itself; 
and shown last are the separate criteria to which the hypothesized 
factor was related. A ''+" in the table indicates that the 
factor contributed to success on the criterion in question, 
and a ''e'' indicates that the factor detracted from success on 
the criterion in question. Tables 16-19 in Chapter VI should 
be consulted to find the strengths of the relationships shown 
in Table 48, and to determine the variable numbers used in the 
study to represent the hypotheses. Further, the discussions 
in Chapter VI concerning Tables 16-19 should be referenced for 


interpretation of the relationships. 
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12 


14 


16 


20 


23 


31 


32 


33 


TABLE 48 


Summary of Findings 


User 
Satis- 
Hypothesis Time Cost faction 

Participation by operat - + 
ing management in design, 
formal approval of 
specifications, and 
continual review of 
project. 


Organization level of 
top computer executive. 


Documentation standards 
used and enforced, 


Low turnover of project 
personnel. 


Source of origination of 
project (MIS staff or 
user). 


Length of experience in 
the organization of 
project personnel. 


High level programming 
language used for 
project. 


High formal educational 
level of project 
personnel. 


Separation of analysts 
and programmers for 
large projects. 


Overall size of organi- 
zation systems staff. 
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Computer 
Operations 
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Hypotheses Not Related to Criteria of Success 


The five hypotheses found to be unrelated to project 


success for the projects in the sample were: 


Rank Hypothesis 
Z Measurable project objectives from 


conception of the project. 


3 Utilization of a project team composed 
of MIS staff and user personnel. 


8 Systems experience of project personnel. 


13 Use of a formalized and regular reporting 
structure on project progress. 


34 Ratio of computer hardware investment to 
total sales or operating budget. 


General Comments Concerning the Hypotheses 


In reviewing the above findings from the study, one 
impression that immediately comes to the fore is the error in 
the evaluations of the information systems professionals who 
ranked the thirty-five hypotheses in terms of contribution to 
project success. More of the hypotheses were confirmed from 
the lower half of the ranking than from the top half. In other 
words, those factors which the information systems professionals 
thought most crucial to MIS project success tended to be rejected 
more than those factors thought least crucial. However, the 
questions and possible explanations arising from this situation 
are more interesting than the situation itself. 

A fundamental benefit to the researcher in conducting 


this study was that the general confusion which surrounds the 
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definition of MIS was brought into clearer focus. That there 
is rather widespread disagreement among those in the informa- 
tion systems community concerning what MIS really is, or ought 
to be, became apparent in analyzing the responses of the 
information systems professionals who attended the Founding 
Conference of Society for Management Information Systems in 
1969 to a request to define what MIS meant to them. The 
different definitions of MIS were almost as numerous as the 
respondents themselves (Dickson, 1970). 

The researcher tried to avoid this confusion from 
the beginning of this study by concentrating on MIS projects, 
and by defining rather explicitly the criterion which placed 
a project in the management information system category. Very 
simply, any project selected for study had to result in infor- 
mation being provided to a manager, at some level, which was 
used by that manager in the decision-making process relative 
to his domain of responsibility. The emphasis throughout the 
study was on looking at projects which had been conceived to 
furnish managers information that would assist them in making 
decisions in a more effective way. 

The conclusion reached during the course of the 
research was that there are at least three different types of 
computer-oriented information system projects which should not 


be lumped together. These three are: 
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1. Data processing projects, where the basic 
objective is to process operational data in 
a more efficient manner by using a computer 
in lieu of manual or accounting machine methods, 

2. Generalized software projects, where the objecte- 
ive is to produce a marketable software package 
for widespread consumption by various users. 
There is no explicit concern here for the specific 
information needs of one or a few users in one 
organization, although the products of such an 
endeavor might facilitate responding to those 
needs after further development, modification, 
or incorporation into a user-oriented system. 

3. Management information system projects, where 
the objective is to provide a manager, or 
managers, with specific information system 
products that will enable those managers to do 
a better job of decision-making relative to 
specific kinds of problems. 

As has already been pointed out, this research has 
focused entirely on category 3 above. However, the specification 
of a definition for the purposes of this study can in no way be 
expected to go beyond the bounds of this study. In other words, 
although a definition of MIS projects has been provided which 


has suited the purposes here very well, it is only one of many 
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possible definitions. The data and results presented in this 
thesis are relative to only one portion of the information systems 
spectrum, and where factors were ranked by others in the field 
as to their importance to MIS project success, other frames of 
reference were possible. Therefore, to say that the conclusions 
reached here serve to reject the general thinking of professionals 
in the information systems field as to what factors are critical 
to MIS project success would be an error. 

With that background, several further comments can be 
made about the findings and conclusions presented here. Since 
the entire focus of the present study has been MIS projects, 
as they have been defined, all of the findings must be viewed 
in that context. The size and scope of projects, how they 
were managed, the interactions that occurred, and measures of 
success must all be considered in the context of the MIS project. 
This means that generalizations beyond this type of project 
should only be made with the greatest caution, if at all. This 
is not to say that the findings here are of no consequence because 
they don't apply to the entire information systems spectrum. 
Rather, other portions of that spectrum must be investigated 
separately, or at least with an awareness that there seem to 
be fundamental differences in what one is looking at. However, 
even this last statement cannot be confirmed until these other 
segments of the information systems spectrum have been analyzed 
in somewhat the same way that MIS projects have been analyzed 


in this study. 
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The User Participation Cluster 


It was definitely concluded from the analysis of 
the data collected in the study that perceived user participa- 
tion is a crucial factor in the success of MIS projects. There 
was a cluster of variables that seemed to represent, essentially, 
the direct and close involvement of users in the development of 
their projects. This cluster consisted of origin of the 
project, reported clarity of objectives, reported specificity 
of actual information requirements, the use of an interfunctional 
project team, and finally, the user participation variable 
itself. However, making this general statement necessitates 
explanation of several points with respect to success and the 
separate variables just cited. 

First, there is an implicit value judgment that user 
satisfaction with a MIS project is the most important success 
criterion. This is based on the nature of the MIS project as 
opposed to other possible project types. Since the primary 
purpose of the MIS project is to provide information to a 
manager to support his decision-making responsibilities, the 
failure of a project to satisfactorily serve this end means 
failure for the project in terms of its essential purpose. 
Being within budget on time and cost, and creating no problems 
for computer operations, are hardly meaningful if the completed 
project is not used or is viewed by users to be inadequate 


for their information needs. This is not an advocacy of 
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complete freedom from budgets or the need to consider how a 

given project will affect computer operations. It is merely 
a value hierarchy of the criteria, with user satisfaction as 
the primary criterion and the other three as important, but 

secondary criteria of success. 

Moving to a closer consideration of the variables 
included in the cluster mentioned above, several important 
aspects of perceived user participation were brought out in 
the analysis of sample data. First, although clear and measur- 
able project objectives would logically seem to be a big 
factor in users getting what they could use, and within time 
and cost expenditures reasonably close to budgets, this was 
not always the case. In fact, there were enough projects in 
the study which did not follow this logical pattern that the 
nature of project objectives was not significantly related to 
any criterion of success. Those projects which did not fit 
the expected pattern were of two types: projects of the "evolu- 
tionary" type, and projects where users knew what they wanted 
to be able to do, but did not understand their own information- 
decision environments well enough to know how to do it. 

In the first type, the “evolutionary” project, 
objectives were often reported to have been rather vague at 
the beginning of a project, but the users worked so closely 
with the project staff that, over a fairly long period of time, 


during which the users progressively learned more about what 
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information they could use and in what ways, results were 
generated with which the users were highly pleased. In the 
second type, where it was reported that there were clear and 
measurable project objectives, but where the users were not 
able to specify precisely what information they needed, or 
in what form, to reach those objectives, a good deal of 
frustration and slippage occurred during project development. 
It would seem that in these cases a project was begun before 
enough spadework had been done by users. In an effort to 
turn something out the project staff, after floundering for 
a length of time, froze requirements which may or may not have 
been what the users ended up feeling they should have gotten 
from the project. 

With respect to the source or originator of a project, 
those initiated by users were generally more successful. The 
users usually knew what they wanted, and were thus able to work 
with project staffs in getting it. However, where users initiated 
a project, but then did not participate very much in the develop- 
ment efforts, results were poor. While the lack of user partici- 
pation may have resulted, in part, because the project leader 
did not allow it, the major problem appeared to be users who 
were unwilling to contribute in a meaningful way. As a conse- 
quence, the project leader made decisions by default which 
affected the acceptability of the final products in the eyes of 


the users. 
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From the preceding comments one draws the conclusion 
that some organizations, and some individuals in organizations, 
have not come to recognize a difference between MIS projects and 
data processing projects. This situation was nowhere more 
apparent than in the findings regarding the use of interfunctional 
project teams. While teams did appear to enhance user partici- 
pation, there were cases where the team approach was implemented 
in such a way that project success was marginal, if not poor. 
There were enough of these cases that project success and the 
use of teams were not significantly related to each other. 

It was found that merely setting up a project team 
with users represented does not insureparticipation of the 
ultimate users of project outputs. In several cases, the data 
processing orientation seemed to dominate. It was assumed by 
user management, erroneously, that merely having someone on the 
project staff from the user area meant there would be high parti- 
cipation, and the user management would end up with satisfactory 
results. This approach may have served quite well for data 
processing projects, where the primary focus was on getting 
procedural aspects of an operational function cut over to the 
computer. A staff specialist from the user area who knew the 
procedures, records, and so forth, could do an excellent job 
for the user function in working as a member of the data 
processing project staff. However, where the project is 


primarily oriented to providing a manager with information to 
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assist his decision making, that manager, not the staff 
specialist, should be the one involved. The fact that in 
some cases this did not occur gave rise to the disagreement 
found among user personnel interviewed about the same project. 
The staff person from the user area felt he had participated 
heavily, and the results of the project were very good, whereas 
a manager, who was actually receiving the project outputs, felt 
participation was not so high and the results were not so good. 
Another problem, related to the one just discussed, 
was the case of the projects where the user representatives who 
worked on the project staff were not the people who actually 
implemented the project. For one reason or another, the user 
representative was given other tasks, not directly related to 
the MIS project studied, soon after the project had been 
completed. This meant that people unfamiliar with what had 
transpired during project development were forced to implement 
the project, and to maintain continuing liaison with the 
information systems function. The results, not unexpectedly, 
were rather poor. The situation just described occurred either 
because the user representatives on the project team were staff 
personnel who were put on something else as soon as the project 
was completed, or the managers who had wanted the project 
developed, and worked with the project team, were transferred 


shortly before or after project implementation. 
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The Project Management Cluster 


Three variables were viewed as comprising a project 
management cluster: documentation, project control, and the 
choice between separate or combination analyst/programmers. 
All three of these factors have been subjects of considerable 
discussion in the information systems literature, as pointed 
out in Chapter II. It would appear that much of the clamor 
surrounding these three factors has arisen as a result of 
the difficulties experienced with the development of data 
processing projects and generalized software projects. However, 
no distinction has generally been made among the different 
kinds of project environments in prescribing what those 
involved in developing computer based systems should do. The 
implicit assumption seems to have been that all projects are 
essentially the same, and, therefore, the same nostrums which 
were believed to be useful in remedying data processing and 
generalized software development ills would be applicable in 
the management information system environment. 

There was certainly no evidence in this study which 
leads one to advocate anarchy in the management of MIS projects; 
which nudges one toward the conclusion that project management 
efforts are futile and should be abandoned. However, there 
was evidence to support the position that MIS project manage- 
ment should be considered in the MIS context. Unfortunately, 


in this regard, about the best the data analysis from this 
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study can do is raise some questions which need to be investi- 
gated thoroughly. 

What are these questions that analysis of the data 
collected have brought to focus? First, is the issue of 
assigning separate analysts and programmers to a MIS project 
versus assigning one person who performs both functions. This, 
of course, does not mean that only one person would be assigned 
to a project in all cases; rather, that the project would be 
divided into functional modules where one individual takes 
responsibility for all analysis and programming involved for 
assigned modules. 

Over the past several years, the sentiment has 
shifted from time to time as to which approach should be used; 
at no time has there been general agreement on which approach 
is best. There have been advocates in both camps. It would 
seem that this issue, as with the others raised in this chapter, 
is one without a universal solution. What is "best" probably 
depends on the organization and information system environment. 

At least for the MIS environments of organizations 
sampled in connection with the research reported here, the use 
of combination analyst/programmers tended to yield the best 
results in terms of user satisfaction with project products. 
This seemed to stem mainly from the ability of the combination 


analyst/programmer to respond more quickly and effectively to 
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user requirements and problems. Where the user was able to 
deal with one person who had the whole picture with respect 
to one or more functional modules of a project, the user 
appeared to be more confident of what he could expect from 
the information systems staff and tended to be less 
frustrated by the implementation problems that arose. 

To say that all MIS projects should be developed 
by combination analyst/programmers would, of course, be an 
unreasonable conclusion to reach. Several organizations had, 
in effect, a dual system, where some combination analyst/ 
programmers were used, but where newer personnel, trainees, 
were also used as programmers only. This approach seems to 
have some merit, in that new people must be trained by some 
means. The question that should be raised, however, is whether 
or not this is the most effective means of training new systems 
people. This issue should be investigated, because it is 
possible that user satisfaction with MIS project results may 
be adversely affected where the separate analyst/programmer 
approach is utilized as a means of training new people. The 
important point is for the management of the information systems 
function to be aware of this possibility, so that safeguards 
can be built into project management to minimize adverse impact 
on user satisfaction with project results. 

A second aspect of MIS project management which would 


seem to require a reappraisal is project control techniques. 
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For the projects in the sample, the project control variable 

was not related to any criterion of success, and it was negatively 
related to the project leader's appraisal of documentation quality. 
These findings could be interpreted in several ways. However, an 
explanation that encompasses all of these findings on project 
control is simply that the methods of project control were not 
conceived nor applied in the MIS context. 

Where stringent progress reporting against planned 
activities was required, that reporting did not, in general, 
result in actual control being exercised. In the case of only 
one project were additional resources committed to enable the 
project staff to achieve the target date for completion. In the 
other cases, progress information seems to have been collected 
for its own sake, with little apparent use made of the information 
for control purposes. Where target dates were revised backwards 
as a result of slippage, this was of value to those in the organi- 
zation who were to be affected by the completed project, but 
such use of supposed control information would not seem to 
constitute the whole realm of project control. 

On the negative side, where stringent project status re- 
porting was required, project staffs seemed to have felt pressure 
to get programs running as quickly as possible. However, where 
they had no additional resources available to them, about all 


project staffs could do to ease the pressure was cut corners 





-179- 


on documentation and implementation preparations, and refuse 

to make changes requested by users. In other words, it 
appeared that project staffs were frustrated by control methods 
rather than helped by them. 

The reader may feel that the application of pressure 
as a means of reaching target dates is useful, nonetheless; 
that without such reporting and pressure, projects could go 
on for longer periods than they already do. And it is 
certainly true that with respect to the projects in this 
sample which were subject to tight progress reporting require- 
ments, the actual times may have been less than they would 
have been had there existed no progress reporting. It is 
impossible to resolve that question, of course. The really 
important point in all of this, however, is the relation of 
the pressure to the quality of the original project estimate. 

It appeared to the researcher that project time and 
cost estimates were probably made relative to data processing 
projects rather than to MIS projects. In very few cases was 
there recognition at the beginning of a project that, in the 
MIS environment, managers must learn to use what they receive; 
that they may want to change the content or format of outputs 
as they go along to whatever is easiest for them to use or 
fits into their decision-making styles. Therefore, as a result 
of poor estimates in the first place, some project staffs were 
subject to pressure to complete projects within artificial time 


frames. 
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Finally, with respect to perceived quality of project 
documentation, and documentation standards, no significant rela- 
tionship was found between these two variables. Although 80% of 
the projects in the sample were subject to some form of documen- 
tation standards, the perceived quality of documentation was not 
assured by such standards. It should be pointed out that a weak- 
ness of the present study was the absence of any objective measure 
of project documentation which would allow the comparison of all 
projects against the same specifications. However, the evaluation 
of project documentation by project leaders was valid to the extent 
that projects free of documentation problems did seem to meet a 
functional test of documentation effectiveness. The assumption 
was made that where there were no documentation problems perceived 
by those who were using the documentation, the documentation met 
the functional test of quality. 

The above conclusions are not intended as a case for 
no standards or no control with MIS projects. Rather, the 
point to be made is that both control and documentation standards 
should be conceived and applied to MIS projects with an explicit 
recognition that the MIS project is different from the data 
processing project or the generalized software project. 
Documentation standards should be oriented to providing clear 
understanding among all those concerned with a project, rather 
than to fixing responsibility for who said what to whom. The 


whole thrust of the standards should be toward making it easier 
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for evolution with users to occur rather than inhibiting it. 
The standards should be such that change is facilitated, rather 
than prevented through freezing requirements too rigidly at 
some arbitrary point. Project documentation is certainly no 
less important because the project is oriented to providing 
information to managers to assist their decision-making. 

This was borne out by the relationships between the perceived 
quality of project documentation and both users' and project 
leaders' views of implementation problems. However, the 
kinds of documentation standards which have been applied do 
not appear to be the most effective for assuring the type 

of documentation needed with MIS projects. 

The same sort of argument holds for project control 
methods. All control should not go out the window because the 
MIS project is somehow different from the kinds of projects 
control schemes have been devised to handle. In fact, it is 
quite possible that MIS project control will provide a 
greater challenge to information systems managers than data 
processing projects have. Where time estimates cannot be 
fixed as easily, and where several projects are running at 
one time in various stages, the allocation and control of 
resources will likely be more difficult. But the point is, 
that if managers are to derive truly usable products from 
MIS projects, they should be allowed to grow and experiment 


with results. This means a more fluid situation where estimates 
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are not based on just the apparent facets of a project at 
original definition. The estimates will also have to be based 
on the kinds of products involved and the individual situations 
of the managers who are to be served. The notion of many open- 
ended projects, current over a long period of time, may seem 

to be an impossible situation with which to live. But what 

is the point of imposing artificial constraints from another 
environment on MIS projects, only to achieve time and cost 


"success'' for results that are ineffective or not used? 


The Project Leader's View of Project Success 


It has been argued here that the most crucial critere 
ion of MIS project success is user satisfaction with the 
results of the project. How do project leaders view project 
success? It would appear, based on the sample data, that 
project leaders implicitly, if not explicitly, understood the 
ascendency of user satisfaction over other criteria for MIS 
projects. Project leaders’ perceptions of project success 
were related very strongly to how satisfied users were with 
project results. Further, project leaders clearly recognized 
the great importance users attached to implementation problems. 
Where users were able to use project outputs with relative 
ease, project leaders felt the projects were successful. Most 
project leaders also seemed to recognize that cluster of user 
participation variables, so important to user satisfaction, 


as important in contributing to project success from their own 
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viewpoints. 

While project leaders did accord time success some 
importance in evaluating overall project success, this was 
secondary to user satisfaction. Further, the project leader's 
view of project success was not significantly related to either 
cost success or computer operations success. In short, project 
leaders were keenly aware, in most cases, of how the users felt 
about projects, and these users attitudes were the most impor- 
tant key in how successful the project leaders felt the projects 
were. 

Differing Views of Users and Project Leaders 

Although users and project leaders agreed very well 
on what made MIS projects successful, they disagreed on two 
important aspects of MIS projects. Project leaders appeared 
to consider implementation problems as those difficulties 
encountered in getting a project completed and operational. 
They tended to focus on cutover as the crucial point, and 
where problems arose in making the cutover, they felt there 
were implementation problems. Users, on the other hand, seemed 
to focus on problems in using the project products as implemen- 
tation problems. They tended to look at implementation as 
what occurred after cutover; those problems experienced in try- 
ing to work with what had been developed to assist them in 


their decision-making roles. 
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This finding may appear to be a contradiction of 
what was said earlier concerning the project leader's percep- 
tions of success, and how those perceptions were strongly 
related to users reporting few implementation problems. However, 
there really is no contradiction. The project leader, nurtured 
in the data processing environment, tended to look upon imple- 
mentation as getting a set of programs up and running. Once 
this was accomplished, and programs were reasonably free of 
obvious bugs, he and his staff went on to something else, 
new projects. Users, on the other hand, had to live with 
what had been given them. No matter how free of technical 
bugs a program was, users had to try to use what they received. 
Where there was no followethrough by the systems staff in 
helping users to do this, users became frustrated and felt 
there were implementation problems, Although project leaders 
seemed to be aware of the frustrations and problems felt by 
users, and recognized these frustrations and problems as 
important factors in how well satisfied the users were, the 
project staffs were often already committed to other projects. 
This meant that although they recognized that follow-through, 
and perhaps changes, were necessary to help users get the kinds 
of information needed, they were not free to do anything about it. 
A second area where users and project leaders disagreed 
to some extent was in evaluating how specific original user 


requirements had been. The project leaders tended to lump 
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specificity of user requirements in with the cluster of user 
participation variables, whereas users appeared to evaluate, 
in retrospect, how clearly they were able to state what they 
wanted at the beginning of the project. 

The evaluations by users of how specifically they 
had stated what they wanted were probably colored by the 
growth process in working with outputs. It appeared that 
users tended to be less satisfied after a period of working 
with a project than when the project was just completed. This 
was interpreted as a reflection of user learning. Thus, when 
the user was interviewed, his feeling that he was not presently 
getting what he would really like to have probably influenced 
his judgment of how clearly he had stated what he wanted in the 
first place. He seemed to assume that if his requirements 
were not now being satisfied as well as he would like, he had 
done a poor job of saying what he wanted. This, of course, 
was not necessarily the case, because the user may have stated 
fairly well what he wanted when he started. But through the 
experience of using project outputs to do his job, his require- 
ments had shifted to some degree. 

The project leader, from his perspective, was 
concerned with trying to pin requirements down so something 
could get programmed, The fact that he had users working with 
him on a team, or readily available to work with his staff if 


not on a team, probably facilitated his getting specific 
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requirements defined. The fact that the project leader sought 
to pin down requirements was natural; it was his job to do so. 
However, the error was in assuming those requirements would not 
shift after the project was implemented. It should not be 
construed that this was necessarily the project leader's error. 
The very nature of most data processing environments precluded 
his going very far beyond implementation. The evolutionary 
approach was not recognized nor built into the development 


process. 
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CHAPTER IX 
GENERAL CONCLUSIONS AND SUGGESTIONS 


FOR FUTURE RESEARCH 


General Conclusions 

The most fundamental conclusion drawn as a result 
of conducting the research reported here is that different 
environments exist for different kinds of information system 
projects. While this may sound like a truism, and a very 
superficial result of an extensive research effort, the impli- 
cations of that conclusion pervade virtually all of the findings. 
If professionals in the information systems field know that 
different kinds of projects do exist, and that they should be 
treated in different ways, that knowledge is not readily 
apparent in either the literature of information systems or 
the practice of those whose vocation is information systems. 

It was stated in Chapter VIII that at least three 
different types of information system projects appear to exist. 
These three types of projects would all seem to place different 
kinds of requirements on those responsible for managing their 
development. The present study was devoted solely to MIS 
projects, so conclusions should not be generalized to other 
types. 

With MIS projects, the primary objective is to provide 


information to managers which they can use effectively in 
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carrying out their decision-making responsibilities. For this 
reason, the satisfaction of the manager with the results of 

the MIS project should be viewed as the most important criterion 
of project success. Time success, cost success, and computer 
operations success, while important, are secondary to user 
satisfaction. 

To achieve success with MIS projects, the findings 
in this study suggest the following approaches to managing 
project development: 

1. Adopt the evolutionary concept of project 
development. This implies recognition of 
a learning process among users which may 
only begin, but is certainly continued, 
after the project has been initially cut 
over and the user is receiving outputs. 
Where the main objective of a project is 
to enable a decision-maker to function 
more effectively, the decision-maker must 
be given the greatest precedence in 
project development plans. Projects should 
be considered from their conception as rather 
fluid, with provision for a trial and error 
process of growth on the part of the user. 
The project should not be viewed, in most 


cases, as a oneeshot development effort 
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which has a specific end point. 

Because of the nature of the MIS project, 
follow-through by the information systems 
staff is imperative. Close involvement 
with the user should not terminate upon 
initial cutover, but should continue to 

be supported by systems staff resources 
over a fairly long period of time. It 
would be expected that this resource 
commitment would decline as the user 

gains experience and confidence with the 
project outputs. Changes to project 
outputs would tend to be greater at first, 
decreasing in scope as the manager sights- 
in on what is really the most valuable 
information to him. 

Project management should be oriented to 

the requirements of MIS projects, as 

opposed to attempting to manage all informa- 
tion systems projects the same way on the 
implicit assumption all types are essentially 
alike. Documentation standards and project 
control techniques should be devised which 
allow evolution rather than thwart it. Time 


and cost estimates should be made in light of 
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the evolutionary concept, and should not 

be used as cudgels to speed program conver- 
sion to free resources for other purposes. 
Wherever possible, combination analyst/ 
progranmers should be utilized to facilitate 
user satisfaction. The ability of the user 

to look to one person who can respond knowledg- 
ably and quickly to his problems is impor- 
tant, and it would appear that the combination 
analyst/programmer is better able to do this 
than individuals who have done only analysis 
or programming work on the project. 

Large projects which cover several functional 
areas should be avoided. Such projects would 
seem to be too inflexible for MIS, and too 

far removed from individual users. Blumenthal's 
(1969) framework for MIS projects would seem 
to be an appropriate model to follow; large 
systems could be broken into small modules, 
which are both more manageable, and more 
meaningful for users. The large underlying 
projects which are needed to support MIS 
projects could be approached as either data 
processing projects or generalized software 


projects. An example of the latter would be 
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a generalized file management system which 
can support many different MIS projects. 

6. User participation means just that; those 
who are participants in MIS project develop- 
ment should be the managers who will use the 
products. The technique of assigning a 
user area staff analyst to the project team 
was fine for data processing projects where 
the primary requirement was for user area 
procedural knowledge and expertise. However, 
the MIS project focuses on the manager and 
his information needs. The definition of 
those needs should not be left to someone 
else if the manager is to be satisfied with 
what he gets. 

The author has no delusions concerning the difficulty 
of following the above suggestions. No specific steps have been 
outlined which make it easy to implement any of these things. 
And, there is no guarantee that the separate suggestions, if 
followed, will bring success. However, the important findings 
of this exploratory study are that organizations should move 
towards different sets of ground rules for MIS projects than 
have been prescribed for data processing projects or generalized 
software projects. The objective of user satisfaction should 


guide the thinking of information systems managers as they seek 
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more effective ways to use the computer in their organizations. 


Suggestions for Future Research 


It is believed that this study has pointed up several 
areas requiring more explicit and intensive research; areas 
which, if researched, should provide some answers to the thorny 
problems inherent in pursuing the course that has been outlined 
above. Among the issues raised here which should be investigated 
thoroughly, and by experimental means, if possible, are: 

1. The kinds of information used by managers in 
making decisions concerning their domains of 
responsibility, and the places they get such 
information now. 

2. The effects of various project control schemes 
on project staffs; the tradeoffs among maintain- 
ing order, dysfunctional pressure, and willingness 
to adapt to user desires which may put the project 
in conflict with the control scheme. 

3. Related to 2 above would be research on the 
reward structures in various organizations 
as they affect systems personnel engaged in 
developing MIS projects. What kinds of organi- 
zational rewards are tied to explicit and 
implicit organizational objectives? 

4. Techniques for allocation and management of 


information systems staff resources where 
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projects do not "end" at an initial, clear 
cutover point. This would encompass techniques 
for estimating project time in the evolutionary 
approach. 

Techniques for integrating managers into the 
development efforts on projects intended to 
provide them with information for decision- 
making. What different approaches are possible, 
and most effective, in getting managers to 
accept the kind of communication required in 
following the evolutionary concept of MIS 
project development? 

If combination analyst/programmers should be 
used to develop MIS projects, how might new 
systems personnel best be trained to minimize 
deleterious effects on user satisfaction? 

What are correlates of project success for other 
kinds of information system projects, such as 
data processing projects and generalized soft- 
ware projects? In these other project environ- 
ments, what should be the criteria of success, 
and how should those criteria be viewed in terms 
of importance? 

What kinds of project documentation are most 


crucial to MIS project success? Is it possible 
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to devise measures of documentation 

quality which can be applied objectively 

to all MIS projects studied to allow 

comparisons across projects and organizations? 
9. Is there a means for evaluating MIS project 

complexity on an objective basis allowing 

comparison across project and organization 

boundaries? 

Most of the above nine areas of suggested research are 
broad. However, if the present research has brought these issues 
into clearer focus so that further research can be conducted 
which will provide meaningful insights into the management of 
management information systems, an important objective has been 


achieved. 
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APPENDIX A 


SMIS QUESTIONNAIRE 


One of the most important areas in the MIS field is 
that of the management of MIS projects. Below are listed a 
number of factors which can influence the overall success of 
MIS projects. We would like to have your opinion concerning 
how these factors influence MIS project success. 

We recognize that "project success" is a vague term. 
For this reason we prefer that you use your own definition and 
frame of reference for making responses. Likewise, "project" 
is vague; so assume that it refers to a development effort on 
the part of EDP and/or user personnel requiring at least six 
man-months to complete. 
Directions: 

Please evaluate each of the thirty-four listed factors 
in terms of the factor: 

"Performance standards employed for analysts 

and programmers" 

In other words, is any given factor cited below - more, less, 
or of equal importance - in determining MIS project success 
than is "performance standards for analysts and programmers." 


Please check the appropriate blank for each factor. 
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Standard factor: "Performance standards employed 
for analysts and programmers" 


Importance compared to standard: 


much somewhat about somewhat much 


cme a ieee ce SE 


1) High formal educational level 
of project personnel. ..... 


2) Proficiency of project person- 
nel as judged by superiors. .. 


3) Length of experience in the 
organization of project 
personel . . . + « + + e.0 « « 


4) Systems experience of project 
WETrSONNe! . . sc « spe @ ¢ @ @ 


5) Coordinating ability of project 
leader (superior's evaluation). 


6) Persuasiveness of project 
leader (superior's evaluation). 


7) Low turnover rate of MIS Dept. 
Staff 6 e e e Cd e e 6 e e e e e 


8) Low turnover of project 
personnel . . . 2. 6 © « © © © 


9) High average income level of 
MIS Dept. Staff e e e e e e e 6 


10) High rates of MIS Dept. Staff 
drawn from within the organi- 
Ps a ne a ee 


11) High availability of computer 
time for program testing. ... 


12) Program maintenance and review 
responsibility specified for 
definite period after imple- 
mentation . . . 6 « « © « «© « « 





Standard factor: 


13) 


14) 


15) 


16) 


17) 


18) 


19) 


20) 


21) 


22) 


23) 


24) 


"Performance standards employed 


for analysts and programmers" 
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Importance compared to standard: 


much 
less 


Measurable project objectives 
from conception of the project 


Formal project selection 
process used to determine 
which projects to develop. .. 


Separation of analysts and 
programmers for large projects 


Combination analyst/programmer 
for small projects ...... 


High level programming lang- 
uage used for project. .... 


Formal training program set 
up for user organization... 


Number of years experience 

for organization with computer- 
ized information systems ... 
Documentation standards used 

and enforced . . ....e.-. 


Utilization of a project team 
composed of MIS and user 
personnel. . . . «© 6 «© © eo « 


Organizational level of top 
computer executive . .... . 


Source of origination of pro- 
ject (MIS staff or user)... 


Participation by operating 
management in design, formal 
approval of specifications, 
and continual review of pro- 
NECCeemicl a) soos a Cetus 0 SY ane 


somewhat about 
less 3 ame 


somewhat 
more 
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Standard factor: 


25) Operating management conducts 
periodic management audit of 
MIS function. (Evaluation of 


26) 


27) 


28) 


29) 


30) 


31) 


32) 


33) 


34) 


"Performance standards employed 


for analysts and programmers" 


effectiveness for users)... 


Utilize existing data base 


versus constructing or 


greatly modifying one. .... 


Short-term, minor project 


versus large, complex project. 


Planning and accounting for 


all resources throughout 


project development ..... 


Utilization of a formal time- 
scheduling technique such as 


PERT for project development . 


Use of a formalized and 


regular reporting structure 


on project progress. .. 


Ratio of computer hardware 


investment to total sales 


or operating budget. ..... 


Overall size of organization 


systems staff. . . . « « «© « « 


Low degree of overall organi~ 


zational change. ... . 


High centralization of organi~ 


zational MIS activities. 


e e 
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Importance compared to standard: 


much 
less 


somewhat 


less 


about 
8 ame 


somewhat 
more 


much 
more 
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Please list any other factors which you believe are important to the 
success of a MIS project. 











S11 
12-18 
£3=23 
24-26 
27=29 
30-31 


32-33 


APPENDIX B 


QUESTIONNAIRE FOR STUDY OF 
SUCCESS/FAILURE CORRELATES IN NIS 
PROGRAMIING PROJECTS 


Environmental Factors 


A. General Information - Company or Division 


we 


Basic Industry: 
a. Manufacturing 

(1) Consumer 

(2) Industrial 
b. Wholesale, Retail Trade 
c. Transportation 
d. Financial 
e. Utility 
f Other (indicate nature) 
If Manufacturing, Primary Production Technology 
a. Unit Job/Small Batch 
b. Assembly Line/Mass Production 
c. Continuous Process 
Company or Relevant Division Size Data 
a. Total Assets 
b. Total Sales 
¢. Total Employees 
d. Number of Billed Customers 
e. Number of Products 
f. Number of Manufacturing Locations 


g.- Number of Distribution Points 
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a 
34 

By 
35-36 

oe 
37=38 

ie 
39 
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Heve there been any significant organizational changes in 
last three years? (i.e., mergers, acquisitions,etc.) 
(Circle one) 
Several 

No signifi- Minor Extensive 
cant changes Changes Changes 

5 4 3 2 } 
What was the ratio of computer hardware investment to last 


year's total sales or other gross income measure? (If 


rented, montuly rental x 40) 





What proportion of the operating expenses of the information 
systems functicn was accounted for by computer hardware last 


year? 





Centralization of organization's informaticn system activities. 
a. If a new MIS application is proposed, which of the 
following statements best describes how the organization 
accomplishes the design, analysis and programming 
activities for that project? (Check one) 
All design, analysis, énd programming 
tasks are performed by the corporate 


MIS staff regardless of who originated 
the project. 5 





All design, analysis, and programming 

tasks are performed by the corporate 

MIS staff only if the project involves 

organizational segments other than the 

one proposing the project. (1.E., If a 

division has its own systems staff, and 

the project involves only tnat divisior's 

information needs, the division will develop 

its own programs within corporate guidelines.) 4 





General system design always done at the 

corporate level but analysis and programming 

performed by the organizational segment that 

wants the project accomplished. 3 
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The corporate level NIS staff prescribes 

standards and procedures for project 

development, approves or disapproves 

project proposals, and evaluates MIS 

progress and performance. Corporate 

staff does not do the analysis and 

programming. 2 





The cerporate MIS staff provides guidance, 
coordination, and assistance to organiza- 
tional segments but all design, analysis 
and programming performed by the organiza- 
tional segment. 1 
8. Level of the top computer executive: 
a. What is the organizational location of the information 


systems function: 


A separate function reporting directly to top management 











4 
A separate function reporting directly to administrative 
40 VP or other similar executive. 3 
An organizational component of some other staff 
function such as the Controller. 2 
An organizational component of a line function 
(specify line function) N 


b. How many hierarchical levels are there between the 


manager of the information systems function and the 


41 top operating executive? (Circle one) 
0 l Z 3 4 or more 
42-44 9. How many personnel are employed by the company in the systems 


design, analysis, and programming activities? 





45 


46 


II. Project Information 
A. Provided by Project Leader or Information Systems ttanager 


1. 


\ho originated this project? 


User departwent 

Top level management 
Information systems staff 
Other (specify) 


Nature of the project: 


a. 


C. 


Cbjectives of tne Project: 


Aa. 


What functional area of the organization was the 
primary beneficiary of the project? 


(If the project had several primary beneficiaries, 
list them all.) 


Project leader's brief description of the nature of 
the project. 


Project completion date: 


Svecific measurable obiectives were Laid down tn 
writing at the outset of the project. 

Specific but non-measurable objectives were laid 
down in writing at the outset of the project. 
General but clear objectives were laid down in 
writing at the outset of the project. 

General objectives were agreed upon by 

key parties involved at outset of project but 

not committed to writing. 

Odjectives of the project were rather vague at its 
initiation but were more fully developed later as 


the project progressed. 
Project objectives were not quite clear to those 


engaged in the project. 
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(Check most appropriate statement) 
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59-51 


52553 


54 


55 
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Project leader's rating of the relative complexity of the 
project: (Check one) 


The most conplex project I have been invoived with 
in this organization 5 


One of the more complex projects we have undertaken 4 
About average complexity 3 


Less complex than nany of the organization's IS 
projects 2 


One of the least complex iiIS projects we have undertaken 1 
Was a project team composed of user personnel and information 
systems staff utilized for this project? Yes 1 
No 0 
If yes: 
a. Were user persounel full tiime or part time on the 
project? Full tine 1: 
Part tirie QO 
b. What proportion of the preject team was made up of 
user personnel? 
c. Of the total man montis devoted to the project, what 
proportion were accounted for by user personnel? 
d. When the design and programming effort was completed 
were the user personnel responsible for implementing 


the project in their respective organizational areas? 


Yes l 
No 0 


e. Did the user personnel who worked on the project team 
carry on tne liaison with the information systems staff 
after implementation? 


Yes 
No 


oe 





56 


Syl 


33--60 
61-62 
63-64 
65-66 
67 


68-69 


70-72 


6. 


Me 


8. 
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f. ‘Yhile on tne project team, to whom dic user 


personnel report as their superior? 





Project leader 3 
Information system mor. 2 
Regular superior 1 
g. By whom were user personnel evaluated? 
Project leader = 5 
Infornation system mer. Z 
Regular superior i 


h. In the opinion of the project leader, hewv instrumental 
were user members in contributing to project success? 
(Check one) 

Very great contribution -- were critical to success 


iiade some contribution; could not have done so 
well without ther 4 


Their presence on the team didn't contribute much; 
could have done just as well without them 3 


Their presence on the team had a sligntly negative 
influence, we had to train them, etc,, slowed us dcwn 
2 
Their presence on the project team was detrimental 
to project success, created conflict 1 
dow many people vrorked directly on the proifect? 
Number of Analysts 
Number of Programmers 
User Representatives 


Consultants 


How many months elapsed from initiation of the project 
to implementation? 


liow many man montiis were spent on the project? 





CARD #2 


2-4 


5-6 


¥O-11 


12 


10. 


ll. 
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Were the analysis and programming functions accomplished ty 


separate groups or by personnel tho were combination analysts/ 


programmers? Combination 


1 
Separate groups 0 


Systers experience of project personnel: 


a. 


How many months experience in systems work has the 
project leader had? 

What is the mean length of syvstens expertence (in 
months) for project personnel, including user 
organization members if anplicable? 

Provide sar. information as for 9 (b) above 
excluding user organization project members. 

What proportion of tue protect staff had two or 
more years of systems experience at the beginning 
of the projsect? 

Project leacer's appraisal of impact that prior 
systems experience of project staff had on project 
success: 

uigh experience was critical to project success. 5 
Experience contrivuted somewhat to project success. 4 


Experience in systems work was not very important 
one way or the other. 3 


Experience had rixed effects; the hizh experience of 
Some teat met vers was very important, but was offset 
somevhat by inexperience of other team i.cmbers 2 


On balance, the lack of systems experience was 
detrimental to project success. : 1 


Project staff turnover: 


a. 


What was the percentage turnover within the project 


staff during the course of the project? 
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(This should be calculated as follows): 


A#D 
2? 





13-14 where: 


A=the number of individuals added to the project 
staff as replacements for departed members 
Dethe number of individuals who left the project 
steff vefore the project's implermentation for 
any reason besides completion of all assigned tasks. 
Psthe median number of individuals on the project staff. 
b. Project leader's appraisal of tie effect of personnel turnover 
on project success: (Circle one) 
15 Very Somewhat lade no Contributed 
cetrimental detrimental difference to success 
4 3 2 1 
12. Project staff educational level: 
16-17 a. What proportion of the project staff held college 
degrees? 
D. ‘What pruportion oi: the projece stasl completed at 
18-19 least to years of college or junior college? 
c. What was the mean nurber of years of education for 
20--21 the project staff? 
(12 years for hish school; 4 years for colleve, etc. 
Person with a college degree would have 16 years of 
education) 
13. Oreanizational expertence of protect staff: 
a. wnat was the mean number of years experience in thts 
22-23 organization for project staff members at the beginning 


of the project? 





24-25 


26 


a3 


28 


29 


b. 
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What proportion of the project staff had two or more 
years of experience in this organization at the Leginning 


of the project? 


14. Documentation. 


a. 


Were formal documentation standards specified for or 


applicable to the project? Yes 1 
No 0 


Project leader’s appraisal of importance of the 
documentation standards: (Circle one) 


Made signifi- 

cant contribu- Somewhat Were just 

tion to success useful make-work 
5 4 3 2 l 


Project leader's appraisal of compliance of the 
project staff witn documentation standards: 


Project staff followed standards completely; each 

phase of documentation reauired by standards was 

completed at the specified time during project 

development. 5 


Project staff followed standards for the most part; 
however, documentation was occasionally lacking in 
quality and timeliness. 4 





Tried to follow standards but just couldn't keep up 
with it and get the programming done, too. 3 


Standards were used for guidance mainly. not followed 
religiously. 2 


Did not follow standards; each project member did what 
he felt was required. l 


How much tine and effort were devoted to enforcing 
documentation standards? (Circle one) 


Very great Moderate Very little 


time & effort time & effort time & effort 


5 4 3 ZL 1 


Project leader's appraisal of quality of the documentation: 
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39 Documentation was of high quality: experienced no 
problems because of inadequate documentation. 4 


Documentation was adequate in most cases, some 
problems of inadequate docuzentation did arise 
but they were very minor. 3 


Documentation problems were frequent but such provlems 
did not seriously affect the success of the project. 2 


Documentation problems were serious; had a detrimental 
effect on success of the project. i 


f. If documentation standards were not followed, was this 
a result primarily of: (Check one) 


Unrealistic standards 
Inadequate standards 


31 Lack of time to document 
Lack of willingness to 
document 

Skip 32-33 Other (specify) 


15. Participation by user organization management in cesign, 
formal aoproval of specifications, and continual review 
of project: 

a. What was the degree of the user organization's active 


participation throuyhout the evolution of this project? 


34 (Circle one) 
They 
Very High Moderate provided what Almost 
preat participation anount was asxed for none 
5 4 3 2 l 


b. What contribution do you feel user participation made to 
the success of this project? 


User participation was definitely an important contributor 


to project success _ 5 
35 User participation probably contributec somewhat to 
project success ___4 


User participation of no real consequence one way or 
the other 3 


The lack of meaningful user participation probably 
detracted somewhat fron project success. 7 





36 


37 


33 


39 


Skip 40-41 


42 


ess 
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The lack of meaningful user participation definitely 
had a detrimental effect on project success. 


How would vou evaluate the quality of the communications 
between your staff and the user organization on this 
project? (Circle one) 





Highly constructive Scmewhat 
anc effective Gooc Adeauate inadequate Poor 
5 4 3 2? ] 


How clearly were user organizations’ cetailed requirements 
and desires specified at the cutset of the project? 
(Circle one) 


Very clearly Generally Not too 
animes a luicit liv clear Acequate elieaz Very Vacue 
> 4 3 2 l 


Was management at the top levels of the user organization 
interested in the project and in directing its evolution? 
(Circle one) 


Extremely interested Interested Disinterested 


and active participaticn but oassive and passive 
5 4 3 


ie 1 


Were user Organizations’ desires for products of the 

project incorporated into the project as it developed even if 
doing so entailed changes to what had already been accomplished? 
(Circle one) 


Always Usually Half the time Seldom Never 
5 4 5 2 ] 


Which statement most nearly matches the situation that existed 
for project control on this project? 


There was a formalized reporting system which required 
that each project member report to the project leader 
his progress against assigned or planned tasks at least 


every month. 


No formalized reporting system was required by the 
organization but the project leader instituted his own; 
each project member reported his progress against 
assigned or planned tasks to the project leader at 


least every month. 
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There was no regular progress reporting required of 

project members. liowever, the project leader was in 

frequent touch with all project rembers and thus kept 

the project status and proyress records himself. ’ 3 


There was no regular progress reporting required of 

project memkers. Project leacer either occasionally 

asked project members how they were progressing or 

left it to project members to inform him if falling 

behird scnedule. 2 


There was no tasa schedule set up against which to 
measure prozress of project staff members. 1 


What programming language was used fer this project? 
Were there operating system or other generalized software 
problems? (Circle one) 


No problems liinor problems liajor problems 
3 2 1 


Provided by User Fersonnel 


? 


a. User organization participation in the project: 


2a. How clearly were your orzenization's detailed 
requirements and desires specified at the outset 
of the project? (Circle one) 


Very clearly Generally Not too Very 
and explicitly clear Adequately clear vague 
5 4 3 fd 1 


b. What was the degree of your organization's active 
participation throughout the evolution of this 
project? (Circle one) 
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It was our Provided 
project: liigh par- Noderate what was Almost 
very great ticipation amount asked for mone 

5 4 3 2 af 


c. Did you desire to have more of a voice in specifying 
and approving the products of the project than you 


actually did have? 


Couldn't 

have had Wanted somewhat Definitely did not 

more voice more of a voice have enough voice 
5 4 3 2 1 


d. How would you evaluate the quality of the communicea- 
tions -etween your organization and the systems staff 
on this project? (Circle one) 


Highly constructive Somewhat 
and effective Good Acequate tradequate Poor 
5 4 3 2 1 


e. Do you feel this project resulted in what you wanted 
Or Uhat che stafi pave you? (Circle one) 


Exactly what About half What the 
ve wanted and half staff gave us 
5 4 3 2 1 


f. Did you make formal arrangements for liaison between 
the project staff and your personnel? 


Yes 1 
No 0 


g-. Did you assign any person or persons in your 
organization responsibility for coordinating the 
project in your Organization? 

Yes 1 
No 0 
h. If answer to (e) was yes, did you relieve 


this person (persons) of normal duties? 


Yes 1 
No 0 
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IIL. Success Criteria 
(A project is considered complete when turned over to cowputer 


operations) 





A. Time 
iS manager or project leader 


lt 
61 
62 
63-64 
65 
66-67 
Zs 
68-70 
71-73 Be 
4. 
74 


a. Was the project completed within the calendar 
time originally allocated for it? 
Yes nF 
No 0 
b. If completed on time, were additional staff 


resources, over what originally were planned, 


required to do so? 


Yes 1 
No 0 
c. If answer to (b) is yes, what percentage of 
the total project effort in man months was 
accounted for ty these additional resources? 
d. slGeeempiitedwon sshedwle, c2s unantictrated 
overtime required to do 80? 
Yes 1 
No 0 


e. If answer to (d) is yes, what percentage of 
total project time in man months was overtime? 
What were the actual nan months required to complete the 
project as a percentage of man months originally allocated 
to the project? 
What was the actuel project completion time as a 
percentage of allocated time? (elapsed time) 
a. Did the project leader believe the time originally 
allocated for the project was realistic? 
Yes 1 
No 0 


b. If the answer to (a) is no, what did the project 
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leader believe to be a realistic time period for 
completion of the project (as a percentaze of the 
time actually allocated - e.g., 120%) 


If the time schedule was not net, what was the primary 
reason given by the protect leaccr? 


(Answer this question on the bacx of preceding page.) 
Relative to past experifence, how successfui does the 
project leader feel this project was in terms of time 
to complete it? (Circle one) 

Highly Above Below Very 


successful average Average averace poor 
5 4 3 2 l 


Cost: 


lilS ranager or Project Leader 


nS 


Was the project completed 'rithin the original cost 
budget for the froject? Yes 
No 
What was the actual project cost as a percentage of 
budgeted cost? 
If the project exceeded budget, what was the primary 
reason for this in the opinion of the project leader? 
Relative to past experience, hcw successful does the 
project leader feel this project was in terms of cost? 
(Circle one) 
Highly Above Below Very 


successful average Averaze  _avcrage poor 
5 & 3 Z 1 


_ 
— 
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C. User Fercepttons of Success of Protect: 
For each primury user oreanization- 
For two levels of management in the user organization 


1. 
6-7 

26 
8-9 

3. 
10-11 

4. 

| 

17-15 

D- 


iow closely do the products of the project match what you 


wanted and expected from the project? (Circle one) 


Got even 

more than Good Satisfactory Warginal Poor 

expected match — match match match 
5 4 5 2 1 


Do you believe the originally stated objectives for the 
project were, Or are in the process of being, satisfied? 


(Circle one) 


For the Not Wot very Definitely 
Definitely most part certain well not 
5 4 3 2 1 


Overall, now satisfied are you with the products of the 


project? (Circle one) 


Kighly Well Products Products 
pleused satisfied acceptable uarsinal Tissatisfied 
5 4 3 2 1 


Have there been implerentation protlems associated with 


this project in your organization? (Circle one) 
Very 
Very minor iloderate Considerable serious 


No problems problems problems cifficulties problems 
5 4 3 2 1 
How well has this project been accepted by your organization? 


(Circle one) 
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Very 
Good Satisfactory Marginal negat- 
Enthusiastically acceptance acceptance acceptance ively 
5 4 3 2 1 
What do you think should have been done to improve the 
effectiveness and value of the project with respect to 
your information needs? 
Do you feel this project is completed? Yes 1 
No 0 
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D. Project Leader's perception of Success of Project: 


1. Lo you believe the originally stated objectives for the 


project were, or are in the process of being, satisfied? 


Zz (Circle one) 
For the Not Not very 
Definitely most part certain well io 
5 4 3 2 1 


2. Overall, how successful do you think this project was? 


(Circle one) 


23 Extremely Very Moderately iot 
successful successful successful certain Marginai 
5 4 3 2 1] 


3. How well satisfied do you feel the users are with the 


products of this project? (Circie one) 


Highly Well Somewhat 
24 pleased satisfied Satisfied dissatisfied ‘Dissatisfied 
5 4 3 2 1 


4. Were there implementation problems associated with this 


25 Project? (Circle one) 
Very 
Very minor Moderate Considerable Serious 
No problems problems problems problems problems 


Skip 26-27 mune 4 3 2 1 
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Computer Operating Cost: 


MS Nanager or Cemputer Operations Manager 


ig 


Nas this project caused problems for computer operations? 
(Circle one) 


Very 
Very minor Noderate Considerable serious 


No problems problems frohlems difficulties provlems 
2 4 3 2 1 


Would you say the costs of operation for this project are 


excessive? (Circle one) 


Definiteiy Frobabiy Not Frovavly Definitely 
not not certain are are 
5 4 3 2 1 


Are you running this project less frequently than the users 
desire? (Circle one) 


Run Whenever Usually run Not Run ouch less Virtually 


desired as desired certain than desired never run 


5 o 3 2 ub 
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UNIVERSITY * Minnesota APPENDIX C 


LETTER REQUESTING PARTICIPATION IN STUDY 


MANAGEMENT INFORMATION SYSTEMS RESEARCH CENTER 
SCHOOL OF BUSINESS ADMINISTRATION 
MINNEAPOLIS, MINNESOTA 55455 


July 9, 1970 


Representative's Name 
Position 

Firm 

Address 


Dear Mr. Representative: 


One of our MIS doctoral students, Dick Powers, is doing 
research on the organizational and procedural correlates 

of success with MIS projects. Dick has reached the point 
where he desires to gather data on twenty MIS projects for 

a statistical analysis of several hypotheses he has set up. 
He hopes to collect the required data by interviewing 
selected people in ten associate firms (two MIS projects per 
firm) during the period 1 October to 31 December, 1970. 


Within the next several days, Dick will be contacting you 

to arrange an appointment of about thirty minutes duration 

to explain more fully the research he is doing, and to discuss 
what will be involved on your part should you agree to partici- 
pate. I believe the research is most worthwhile and hope you 
will elect to participate. 


Since Dick will be doing all of the data collection and 
analysis himself, it is appropriate that you know something 
about his qualifications and background. To this end, a 
brief biographical sketch is attached. 


Thank you for your continued support and cooperation. 


Sincerely, 


Gary W. Dickson 
Associate Professor 


GWD/ jls 
Enclosure 
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Brief Biographical Sketch of Richard F. Powers 


Dick Powers is currently a doctoral student majoring in 
management information systems at the University of Minnesota. 
He entered the MIS program in the fall of 1968 and has now 
completed all course work required for the PhD. Dick is a 
lieutenant commander in the Supply Corps of the U.S. Navy, 
and is being sponsored in his doctoral studies by the Navy. 

He holds a BA degree from Rice University, 1958, and an MBA 
from Michigan State University, 1965. 


Since his commissioning as an ensign in 1958, Dick has held 
the following positions up to the time he entered the MIS 
program at Minnesota: 


1959-1960 Supply and Disbursing Officer, USS MASSEY 
(DD-778) 

1961 Aide to the Commanding Officer, Naval 
Supply Center, Norfolk, Va. 

1962-1963 Director of Data Processing, Naval 
Supply Center, Norfolk, Va. 

1964 Control Division Officer, Royal Canadian 
Naval Supply Depot, Victoria, British 
Columbia 

1965 MBA program, Michigan State University 

1966-1968 Director of Computer Training, Navy 


Supply Corps School, Athens, Ga. 
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UNIVERSITY °° Minnesota APPENDIX D 


BRIEF DESCRIPTION OF STUDY 





MANAGEMENT INFORMATION SYSTEMS RESEARCH CENTER 
SCHOOL OF BUSINESS ADMINISTRATION 
MINNEAPOLIS, MINNESOTA 55455 


Research on the Correlates of MIS Project Success 


Nature of the Research: 

An empirical investigation of sixteen organizational 
and procedural factors hypothesized to be correlates of success 
with MIS projects. 


Methodology: 
Study two MIS projects in each of ten organizations 


with as wide representation across industries as possible. 


Criteria of Success for a Project: 
Time 


Cost 
User satisfaction 
Computer operating problems 


Data Collection: 

By structured interview that follows a pre-tested 
questionnaire. The responses are anchored, and multiple 
scales are used wherever possible to improve reliability 
of the results. 


For each project selected for investigation the 
following individuals will be interviewed with the 
interviews requiring approximately the amount of time 


shown: 
Individual Time 
Project Leader 2 Hours 
User Management, Level 1 4 Hour 
User Management, Level 2 % Hour 
Project Total 3 Hours 
For Two Projects 6 Hours 


In addition to the above interviews for each of two 
projects in each organization, interviews will be conducted 
with the following individuals to collect computer operat- 
ing information or information not specifically related to 
one project: 
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MISRC Associate + Hour 
Manager of MIS + Hour 
Manager of Computer Operations * Hour 

1s Hours 


Total Interviewing Time 
per Organization 7* Hours 


Data Analysis: 
All data collected will be analyzed statistically 


using non-parametric techniques such as X“, Qualitative 
analysis of various moderator variables will also be made, 


Expected Results of the Research: 
1. Confirmation or rejection of the numerous 


prescriptions appearing in information systems literature. 


2. A ranking or other classification of those 
organizational and procedural factors related to MIS project 
success in the operational, real-world environment. This 
ranking or classification will have more general meaning for 
MIS management than previous case studies. 


3. The manager should be able to judge more 
accurately which organizational and procedural factors are 
most critical to project success given a particular set of 
circumstances. 


Assurances to the Participating Organization: 


1. Reported results of the research will be made 
available to the participant company. 


2. No company names nor non-public financial 
information will be reported without the express prior 
permission of the company. 


3. No information provided by an individual will 
be divulged to any other person or be attributed to the 
individual in the report without the express prior consent 
of that individual. 


4. All information gathered will be retained 
personally by the researcher at all times. 
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APPENDIX E 


VARIABLES INCLUDED IN THE STUDY 


Variable Variable Source 
Number Name Page Question(s) 


Variable Description and Scoring 
l ASSETS 206 3a 


Collected as a measure of relative 
size of organizations in the sample. 
Asset data were not available for 
three of the organizations. 


Z S ALES 206 3b 


Total sales or other gross revenue 
measure for the past fiscal year. 
Not available for two of the organi- 
zations in the sample. 


3 EMPLOYEES 206 3c 


Number of employees in domestic 

operations of each organization. 
The only measure of organization 
size which was available for all 
organizations in the sample. 


4 CUSTOMERS 206 3d 


Total billed customers. Collected 
as a measure of relative organiza- 
tion size. Not available for two 
organizations, 


» ORGANIZATION CHANGE 207 4 


Degree of organizational change 
during past three years; scaled 
from ''no change" at high end (5) 

to "extensive change" at low end 
(1). Pertains to changes affect- 
ing the whole organization as 
opposed to intrafunctional realign- 
ments. 
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HARDWARE INVESTMENT/SALES 207 5 


Measure of investment in computing 
hardware and supporting equipment 
relative to total revenue for past 
fiscal year. If owned, equipment 
valued at replacement cost or 

current list price for identical 

or comparable equipment. If rented, 
monthly rental price multiplied by 40. 


HARDWARE EXPENSE/BUDGET 207 6 


Represents the proportion of last 
fiscal year's budget for the 
information systems function that 
was accounted for by hardware cost 
(depreciation and/or rental). 


CENTRAL IZATION 207 7 


Centralization of analysis, design 
and programming activities in the 
organization. Scaled from complete 
centralization (5) to decentraliza- 
tion (1). This variable was not 
adequately tested by the present 
sample; eight organizations were 
scered at 5 level and remaining two 
were scored at 4 level. The latter 
two were not at the 5 level solely 
because of small operations research 
groups which did their own design 
and programming. For this reason, 
Variable 8 is not tabled with other 
variables in the study. 


LEVEL OF INFORMATION SYSTEMS MANAGER 208 8b 


Number of hierarchical levels between 
the manager of the information systems 
function and the ton operating execu- 
tive of the organization. Scaled 
from no intervening levels (0) to 

four or more intervening levels (4). 
The lower the score on this variable, 
the higher the information systems 
function in the organization. 
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NUMBER IN SYSTEMS & PROGRAMMING 208 9 


This variable, the total number of 
personnel engaged in systems analy- 
sis, design, and programming activi- 
ties within the organization, is 
used as the numteator for computing 
the value of Variable 61. 


ORIGINATOR 209 ji 


This variable identifies the primary 
originator of a project being studied. 
It is scaled from user origination 
(4) to information systems staff 
origination (2); there were no 
projects reported as "other" (1). 

In terms of rankings, high rank is 
given to user originated projects; 
middle rank to those of top manage- 
ment origin; and low rank to systems 
staff originated projects. 


MEASURABLE PROJECT OBJECTIVES 209 3 


The project leader's perception of 
nature and clarity of project objec- 
tives. Scaled from highly specific 
and measurable objectives in writ- 
ing (6) to very vague objectives (1). 


COMPLEXITY 210 4 


Project leader's perception of the 
complexity of this project as compared 
with others the organization has under- 
taken. The higher the scale value for 
this variable, the greater the complex- 
ity, with most complex scored 5 and 
least complex scored l. 


PROJECT TEAM 210 5 


Dichotomous variable scored 1 if a 
project team including user personnel 
developed the project, and scored 0 
if there was not a project team com- 
prised, in part, of user personnel. 
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17 


18 


19 


20 
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USER MAN MONTHS/TOTAL MAN MONTHS 


The proportion of man months spent on 
the project accounted for by user 
personnel. Variable 15 value will 

be 0 for all projects where value of 
Variable 14 is 0, indicating a project 
team including users was not employed. 


USER CONTRIBUTION TO TEAM 


Project leader's perception of the 
contribution of user members of the 
project team to project success. 

A value for only 13 projects that 
used team. Not included in the 
statistical analysis. 


NUMBER ON PROJECT 


Total number of personnel, including 
users and consultants, if any, who 
worked directly on the project at 
any time. 


NUMBER OF ANALYSTS 


Number of analysts who worked on 
project. Not included in the 
statistical analysis. 


NUMBER OF PROGRAMMERS 


Number of programmers who worked on 
project. Not included in the 
statistical analysis. 


NUMBER OF USERS 


Number of user personnel, if any, 
who worked directly on project. 
Not included in the statistical 
analysis. 


ELAPSED MONTHS 
Number of months included in period 


from formal start of project to pro- 
ject completion; project completion 
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defined as date when project turned 
over to computer operations for 
production running. 


MAN MONTHS 211 8 


Number of man months spent on project 
by all personnel directly involved 
with project, including analysts, 
programmers, user members of project 
team, and consultants, if any. 


COMBINATION ANALYST/ PROGRAMMER 212 2 


Dichotomous variable scored 1 if 
combination analyst/programmers 
were used on the project, and 
scored 0 if the analysis and 
programming tasks were performed by 
separate individuals or groups. 


SYSTEMS EXPERIENCE OF PROJECT LEADER 212 10a 


Total number of months systems 
experience the project leader 
possessed at the start of the 
project. Systems experience 

defined to include programming or 
systems work performed outside of 

a data processing organization, such 
as methods and time studies. This 
variable not used in the statistical 
analysis; Variable 27 used as the 
systems experience variable in all 
reported cross-tabulations. Variable 
24 was highly correlated with Variable 
27, the tau value being .418, which 
is significant at the .014 level 
(two-tailed). 


MEAN SYSTEMS EXPERIENCE OF PROJECT 
PERSONNEL INCLUDING USERS 242 10b 


Mean number of months systems exper- 
ience for entire project team, 
including user personnel and consul- 
tants, if any. Comments under Variable 
24 concerning definition of systems 
experience and tabulation apply 

fully to Variable 25. Tau value for 
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Variable 25 vs. Variable 27 was 
.596, which is significant at the 
.0006 level (two-tailed). 


MEAN SYSTEMS EXPERIENCE OF PROJECT 
PERSONNEL FROM MIS STAFF ONLY ZZ 10c 


Mean number of months systems experi- 
ence for those personnel from the 
information systems function who 
worked on the project. This value 
the same as the value for Variable 
25 for those seven projects where a 
team not used, Comments under 
Variable 24 concerning definition 

of systems experience and tabulation 
apply fully to Variable 26. Tau 
value for Variable 26 vs. Variable 
27 was .575, which is significant 

at the .0004 level (two-tailed). 


TWO OR MORE YEARS SYSTEMS EXPERIENCE Ze 10d 


Proportion of the project staff, 
including users and consultants, if 
any, with two or more years systems 
experience. Systems experience 
defined exactly as for Variables 
24-26. This measure of systems 
experience considered to contain the 
least distortion from extreme values. 


IMPACT OF SYSTEMS EXPERIENCE 212 10c 


Project leader's opinion of the 
impact that prior systems experience 
of project personnel had on project 
success. Scaled from highly critical 
to success (5) to a lack of systems 
experience detrimental to success (1). 
This variable not tabled in the 
statistical analysis. 


TURNOVER 212 lla 


The percentage turnover of the project 
staff. Any person who left the 

project staff at any time prior to 
completion of the project was considered 
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as contributing to turnover unless 
that person's assigned tasks had 

been completed. If a person who 

left the project prematurely was 
replaced, the replacement was also 
considered as a factor in the numera- 
tor of the turnover computation. 

The denominator of the turnover 
computation was the ''normal" or median 
number of personnel on the project 
staff multiplied by two. 


COLLEGE DEGREE 25 12a 


Proportion of project staff that 
held a college degree. Although 
not used as the primary measure of 
the level of education, Variable 30 
did make a contribution to the 
interpretation of certain relation- 
ships, and for this reason it has 
been cross-tabulated with other 
variables where significant 
relationships existed. 


TWO YEARS COLLEGE 213 12b 


Proportion of project staff that 
had completed at least two years 

of college. This proportion 
included all of those who held 
college degrees, and was, therefore, 
very strongly associated with 
Variable 30. Since Variable 31 
contributed nothing to the study 
which was not picked up by using 
Variables 30 and 32, it is not 
tabled in the statistical analysis. 


MEAN YEARS FORMAL EDUCATION 203 Age 


Mean number of years formal educa- 
tion for the project staff, including 
users and consultants, if any. This 
variable used as primary measure of 
level of education for project staff. 
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MEAN YEARS ORGANIZATION EXPERIENCE 238 l3a 


Mean number of years experience in 
the organization (firm, not function) 
studied for project staff members. 
Not included in the statistical 
analysis since Variable 34 believed 
to be a less distorted measure of 
organization experience. 


TWO OR MORE YEARS ORGANIZATION 
EXPER TENCE 214 13b 


Proportion of the project staff, 
including users and consultants, 
if any, possessing two or more 
years experience in the organiza- 
tion (firm, not function) studied. 
This variable the primary measure 
of organization experience. 


DOCUMENTATION STANDARDS 214 14a 


Dichotomous variable scored 1 if 
documentation standards were 
prescribed for the project, and 
scored 0 if such standards were 
not prescribed. 


QUALITY OF DOCUMENTATION 214 l4e 


Project leader's appraisal of the 
quality of documentation for the 
project. Scaled from high quality 
(4) to serious problems with 
documentation (1). 


STANDARDS APPRAISAL 214 14b 
14c 

The sum of two separate perceptions 

by the project leader; how important 

he felt the documentation standards 

were plus how well the project staff 

complied with the standards. Since 

Variable 37 made no contribution to 

the analysis of documentation stan- 

dards or quality, it is not tabled. 








38 


39 


40 


41 


42 


SPECIFICITY OF USER REQUIREMENTS -=-PL 


Project leader's perception of the 
clarity and specificity of user 
requirements and desires at the 
beginning of the project. Scaled 
from very clear and specific (5) 
to very vague (1). 


USER MANAGEMENT INTEREST 


Project leader's perception of the 
interest and participation of top 
level user management in the develop- 
ment of the project. Scaled from 
very active participation (5) to 
disinterested (1). 


CHANGED AS REQUESTED 


Project leader's perception of the 
response by the project staff to 
user requests for changes in any 
aspect of the project during the 
development period for the project. 
Such changes may have entailed 
merely output format modifications 
or may have required major logic 
alterations. Scaled from always 
made requested changes (5) to 
never made such changes (1). If 
no changes of any kind were requested 
by the user, scored 5, 


USER PARTIC IPATION--PL 


Project leader's perception of the 
level of user participation in the 
project. An aggregate variable 
representing the sum of three separ- 
ate items. Scaled from very great 
participation (15) to very little (3). 


PROJECT CONTROL 


Project leader's rating of the type 
and frequency of progress reporting 
for the project. Scaled from formal- 
ized, monthly progress reporting (5) 
to no progress reporting (1). 
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HIGH LEVEL PROGRAMMING LANGUAGE 2a07 7 


Projects programmed in COBOL were 
scored 3; those programmed in FORTRAN 
were scored 2; and those programmed 

in some assembly language were scored 1. 
For the purposes of this study, COBOL 
was considered a higher level language 
than FORTRAN, 


GENERALIZED SOFTWARE PROBLEMS 217 18 


Project leader's perception of the 
existence and seriousness of operating 
system or other generalized software 
problems with respect to this project. 
Scaled from no problems (3) to major 
problems (1). 


SPECIFICITY OF USER REQUIREMENTS -- 
USER 2 la 


Users‘ perception of the clarity and 
specificity of user requirements and 
desires at the beginning of the 
project. Scaled from very clear and 
explicit (50) to very vague (10).* 


USER PARTICIPATION--USER ZA lb 

218 le 
Users’ perception of the level of ld 
user participation in the project. le 


An aggregate variable representing 

the sum of four separate items. 

Scaled from very great participation 
(200) to very little participation (40).* 


ON TIME 219 la 


A dichotomous variable scored 1 if 
the project was completed within the 
calendar time originally estimated, 
and scored 0O otherwise. Variable 47 
not used in cross-tabulations since 
Variable 48 was the primary time 
success variable used in the study. 
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ACTUAL TIME/ESTIMATED TIME 219 3 


The ratio of actual calendar months 
spent on the rpoject over calendar 
months estimated for the project. 
This variable is the criterion 
variable for project success in 
terms of time. 


TIME SUCCESS --PL 220 6 


Project leader's opinion of the 
success of this project, in terms 
of time to complete it, relative 

to his experience with actual time/ 
estimated time for other projects. 
Scaled from highly successful (5) 
to very poor (1). 


WITHIN BUDGET 220 l 


A dichotomous variable scored l 

if the project was completed with- 
in the original cost budget for 
the project, and scored 0 other- 
wise. Variable 50 not used in 
cross-tabulations since Variable 
51 was the primary cost success 
variable used in the study. 


ACTUAL COST/ESTIMATED COST 220 


The ratio of actual cost of the 
project over estimated or budgeted 
cost for the project. This variable 
is the criterion variable for success 
in terms of cost. Where cost data 
were not available, this ratio 
computed based on actual and estimated 
man months for the project, along 
with an estimated factor for computer 
test time. 


COST SUCCESS-=-PL 220 4 


Project leader's opinion of the 
success of this project, in terms 
of cost, relative to his experience 
with actual cost/estimated cost for 
other projects. Scaled from highly 
successful (5) to very poor (1). 





53 


54 


55 


56 


ay 


IMPLEMENTATION PROBLEMS =--USER 


Users’ perception of the problems 
experienced in implementing the 
project in their organizations. 
Scaled from no problems (50) to 
very serious problems (10).* 
Although this variable is actually 
a component of Variable 54, it is 


also treated separately for compara- 


tive purposes. 
USER SATISFACTION--USER 


Users' perception of the success of 
the project in terms of five sepa- 
rate factors which were summed to 
derive one overall measure of user 
satisfaction. Scaled from high 
user satisfaction (250) to low 

user satisfaction (50).* This 
variable is the criterion variable 
for success in terms of user satis- 
faction. 


SATISFACTION OF OBJECTIVES -=-PL 
Project leader's perception of the 


degree to which project objectives 
were satisfied. Scaled from defin- 


itely satisfied (5) to not satisfied 


(1). 
PROJECT SUCCESS--PL 


Project leader's global perception 
of the overall success of the 
project. Scaled from extremely 
successful (5) to marginally 
successful (1). 


USER SATISFACTION--PL 


Project leader's perception of 
user satisfaction with the 
products of the project. Scaled 
from users highly pleased (5) 

to users dissatisfied (1). 
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58 IMPLEMENTATION PROBLEMS-- PL 223 4 


Project leader's percection of the 
problems encountered in implementing 
the project. Scaled from no problems 
(5) to very serious problems (1). 


59 COMPUTER OPERATIONS PROBLEMS 224 1 


Perception of the manager of computer 
operations, or his superior in some 
cases, of the problems caused by the 
project for computer operations. 

Scaled from no problems (5) to very 
serious problems (1). This variable 

is the primary criterion for project 
success in terms of computer operations. 


60 COMPUTER OPERATIONS COST 224 2 


Perception of the manager of computer 
operations, or his superior in some 
cases, of the computer operating 

cost for the project. Scaled from 
cost definitely not excessive (5) to 
cost definitely excessive (1). 


61 SYSTEMS STAFF/TOTAL EMPLOYEES Computed 
from 
A measure of the relative size Variables 3 
of the organization's systems and 10 


staff (analysts and programmers). 
Computed by dividing the total 
number of analysts and programmers 
(Variable 10) by the total number 
of domestic employees of the 
organization (Variable 3). 


* Note: All variables representing users’ perceptions 
were scaled up by 10° to avoid using fractions 
where there were two or more responses per 
item which were averaged to derive one value 
for that item. Such scaling, of course, had 
no bearing on the relative rankings which were 
used in the analyses. 
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APPENDIX F 


COMPUTING FORMULAS usED! 


The following procedures and formulas were used in 
computing the values of tau-B, tau-C, S, variance of S, and 
the normal deviate of S: 

1) All observations on two variables being cross-~ 
classified were entered into a naturally ordered 
table of the following type: 


j#l to c columns 






175 

150 

i=l to r 130 
rows 92 
58 

47 

40 


In the above example case, there are five possible 
levels for one variable, and these are represented by 
the columns. There are seven possible values for the 
second variable and these are represented by the rows. 
Any one observation of the total N observations must 
fall in one of the cells of the resulting array. Thus, 
ieee of what is presented in this appendix is taken from 


the appendix of UMST 590, the computing program used for the 
data analysis, which, in turn, is from Kendall (1962). 
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the entry in each cell is merely the number of 
observations falling in that cell, or having the 
values of variables one and two that intersect in 
that cell. 


2) S = P = Q 


where P s > > x, a 
isl jzl  kzitl Pi 1) Se 


-1 
Qe > _ xX 
isl j22 ksitl i i} ké 


3) For square tables (where ric), Kendall's tau-B was 


rt 
J 
i 
2) 
rt 


computed as follows: 


tau-B = S 

\/ §N(No1) T \VEN(NoL) <U 
where T and U are factors representing the number 
of ties in the ranking of each variable, and are 
computed as follows: 


12> 20, 


isl 


J 
ie 


(nl) 


SG 
| 


1/2 Dia. ja. j-1) 
jzl 


4) For rectangular tables (where rfc), Kendall's tau-C 
was computed as follows: 
tau-C = 2S 
cere, 


where m is the smaller of r or c. 
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5) The normal deviate of S can be computed only after 
the variance of the § statistic has been determined 


as follows: 


Vale “ cost NSIT A, (my DEES) 


a ns gine j=) Qn. 5+5)| 


jie 


(a5 

see Le __| marin lsh) 

+ ON(N=1) (N-2) - i, i. in 
i=l 


an yboe ebOo. 2 


Cc 
l n(n, -l) n. ,(n. .-1) 
mF aes Eales jo 3 | 


121 $21 
6) The normal deviate of S can next be computed by: 
Z(s) 8 S* 
V var $ 
*Before computing the normal deviate, S is 
corrected for continuity as follows: 
a) for a 2 x 2 table, |s| reduced by kN 
unless S < §N, in which case S = 0. 
b) for all other tables, s | is reduced by l. 
These corrections for continuity are made to 
adjust the distribution of S closer to the normal 


distribution where there are ties in either or 
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both of the rankings being cross-classified. 
However, where the extent of tied ranks is 
very great, the correction in (b) above may 
not be enough. This is particularly true 
where one ranking is a dichotomy and the other 
ranking has ties of varying extents in it. 

In these last cases, an actual distribution of 
S for each set of variables cross-classified 
would need to be determined to arrive at the 


1 
continuity correction required. 


hon discussions and proofs of the correction for 
continuity applied to S under various cases of tied ranks, 
see Burr (1960), Kendall (1947; 1962), Sillitto (1947), and 
Whitfield (1947). 
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APPENDIX G 


COMMENTS OF USER MANAGEMENT 


Each user interviewed was given the opportunity at 
the end of the interview to make whatever comments he felt 
were germane to the project being studied, Specifically, each 
user was asked what he felt should have been done to improve 
the effectiveness and value of the project with respect to his 
information needs. Further, the user was asked to elaborate 
on any aspects of the project which he felt required additional 
development work. 

The comments included in this appendix provide some 
flavor of the attitudes of user managers toward the projects 
studied. Since verbatim notes were not taken by the researcher, 
the comments presented here are best described as "paraphrased 
user comments". However, the comments are presented in the form 
of statements by the users rather than descriptive narratives 
of the researcher. 

1. Problems that were apparent from the beginning 
have never been corrected. There has been 
virtually no follow-through by the project 
staff to help us work with the system. We 
have also been unhappy with the validity of 
the input data; we don't consider the reports 


we're getting to be very reliable. 
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We have learned a great deal from the 
experience of working with the system. 

We now feel there is too much historical 
information provided in the outputs; 
completed transactions should be purged 

from the files so that they do not appear 

in our reports. Also, there has been a 

lack of training for the people preparing 
the input data. This has resulted in too 
many input errors which makes us suspicious 
of the validity of the outputs. 

There was too much pressure from top manage- 
ment to get something going in a hurry to 
meet an immediate problem. This led to 
inadequate planning and preparation. I 
think we are trying to do too much with the 
system; we should reduce the scope of this 
project and focus just on the heart of the 
problem, rather than trying to do everything 
for everyone with one project. 

We want a good deal more now than we envisioned 
needing when the system was designed. The 
original outputs were fine, based on what we 
said we wanted, but we have been unable to 


get changes made that we feel are necessary now. 
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Another problem was that the salesmen who 
provide the input were never involved in 
system design. As a consequence, they 
cannot see any results from the coding 

they are doing--theysee no specific outcome 
and tend to feel they are wasting their time. 
This is a very valuable application. The 
opportunity to evolve with it and to incor- 
porate needed changes in phase two was 
crucial to our getting useful outputs. 

We are just getting too much paper. We 

need reports oriented to the needs of the 
individual decision maker. 

From our standpoint, the system is not 

worth too much. The production engineers 
who are supposed to use the system had 

little or no voice in designing it. Higher 
level managers were involved in defining 

the system, which we believe resulted in 
outputs more oriented to accounting needs 
than to production requirements. The 

reports are too voluminous and hard to 

use; there should be more exception reporting. 
I get a lot of performance figures, but these 


are meaningless to me without some standard 
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or scale for comparison. What is good, bad, 
acceptable? I don't believe there is a 
real means for a manager in my position to 
use the system--to really be an effective 
influence on actions taken. Although some 
of the other managers were involved in 
developing the system, I don't believe I 
had enough voice in it. 

9. We don't get the output in a format that 
is usable. Our analysts still have to 
rearrange the data and prepare the required 
reports by hand. However, I think the 
blame for this situation falls on us--we 
did not state explicitly enough what was 
needed, 

10. Eighty to ninety percent of our information 
requires output formatting of a certain type. 
We don't get that type of output at all. The 
need for this type of output format was very 
clearly specified in the original project 
request. 

ll. It seems that the original objectives were 
satisfied well enough, but it is apparent now 


+ Cbmaaece 9 and 10 came from two different users of the 
project outputs in the same organization. 
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that we did not really know what we wanted 

or needed when we started. We have grown 
with experience with the present system--we now 
have a better grasp of what information we 
should be getting but are unable to get it 
with the present system. Among the specific 
shortcomings, as I see them, are: the 
information is not now current or accurate 
enough to make the kinds of decisions we 
should be making; reports are too detailed 
and voluminous--we need much more analytical 
reports that satisfy our information needs 
directly without further manipulation; and 
too many people are involved in data prepara- 
tion with no effective input quality control. 
The form required to be filled out for an 
inquiry run is rather complex. As a result, 
some potential users have not used the system 
as they could. I guess this may partly stem 
from the fact that all potential user areas 
were not well enough represented and involved 
in original system definition. 

Due to the way the project was developed, and 
the way it is now used, the field sales force 


is very opposed to the forecasting system. 
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They were not consulted in the development 
efforts, and they have no input to the processing 
cycle now. However, I guess there has been a 
time constraint that precluded our including 

the field sales force in the processing cycle. 
Very simply, we did not ask for enough; we 

did not specify what we wanted clearly enough; 
and the result is that we are still doing too 
much manually. 

I can think of a couple of things that are 
probably related. First, the documentation has 
been inadequate in that changes to programs 

have often not been reflected in documentation 
nor communicated to users. Second, the field 
training of operating people has been inadequate 
which, I think, accounts for some of their resist- 
ance to the system. 

There are too many inaccuracies in the data-- 
inadequate input controls and data validation. 
Also, the reports are too detailed and difficult 
to use for vice presidents and assistant vice 
presidents. Finally, we have had implementation 
problems. We are not getting the reports where 
and when we should in some cases. This, combined 


with inaccuracies, is frustrating. 
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